Hello Gambit and Snow users. I will be applying to the Google Summer of Code program as a mentoring organization and propose projects for advancing the development of Gambit, Snow, and Termite. I'd like to start a collective brainstorming exercice to come up with a list of specific projects.
If you are interested in participating as a student, please respond to this message (preferably on the mailing list). Please describe briefly what you would like to work on.
If you cannot participate (e.g. you are not a student) but have project suggestions, then please speak up on the mailing list.
Some projects ideas that come to mind:
- native thread support - real-time/parallel GC - closer integration of Termite and Gambit - Gambit packages - web framework - documentation - François Pinard's projects (see MSLUG mailing-list archive)
Marc
Afficher les réponses par date
On Wed, Feb 27, 2008 at 6:12 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
If you are interested in participating as a student, please respond to this message (preferably on the mailing list). Please describe briefly what you would like to work on.
I plan to apply. I would primarily like to work on native threads, and since (ATM) I only have access to multicore/SMP hardware on the sparc/solaris architecture I plan to investigate if the sparc/solaris-port needs work.
Some projects ideas that come to mind:
- documentation
Documentation-only work seems to be off-limits: http://code.google.com/soc/2008/faqs.html#0.1_doc_proposals .
/Joel
On 28-Feb-08, at 9:11 AM, Joel Borggrén-Franck wrote:
- documentation
Documentation-only work seems to be off-limits: http://code.google.com/soc/2008/faqs.html#0.1_doc_proposals .
By "documentation" I don't simply mean "to write documentation". That is one aspect, but just as important is the design of tools and procedures to write, access and display/print documentation. For example, it would be interesting to have a tool to check, for each release, that the Gambit manual's examples are consistent with the interpreter and the compiler. The file doc/checkdoc.scm is a first attempt at this, but it is more of a hack and lacks robustness.
Marc
(I've stripped the snow and mslug list addresses, but added François Pinard because of the Pymacs reference.)
(I'm not a student.)
If you cannot participate (e.g. you are not a student) but have project suggestions, then please speak up on the mailing list.
Some projects ideas that come to mind:
- native thread support
- real-time/parallel GC
It's difficult for me to say anything about threading. I think it's a challenging topic (and I've got no multiprocessor programming experience).
There are two things that I think could be useful for someone like me who was thinking about using threading on top of the Gambit core (as libraries), maybe this will also be helping integration with whatever grows up in the core:
- it would be useful to have a little portable framework for synchronization of memory contents (barriers) and atomic ops, including documentation. I guess gcc 4.2(.2+) may already offer all we need, but maybe alternatives for users of older gcc's or non-gcc compilers could be collected, and especially the knowhow about correct memory handling, and the cost patterns involved would be useful (with comparisons between different architectures, esp. between NUMA (e.g. multi-Opteron) and uniform memory access (SMP like Athlon 64 X2 or FX or Intel Core 2) machines).
- maybe a possibility to integrate mutexes into the Gambit scheduler (especially Linux' futex'es, which may already be used by the pthread mutex implementation in glibc now); i.e. make it possible for a Gambit green thread to block on such a mutex without also blocking other green threads. On top of that, one could implement an extension of the FFI (even as a library) which does FFI calls "into" a separate pthread, in other words: the Gambit green thread would kick off a (potentionally blocking) call in another (possibly pooled) pthread, and wait on the mutex for completion, during which time other green threads are being run by the Gambit scheduler. That's achieving the same thing as using pairs of unix pipe(2)'s or a socket(2) pair (and writing the procedure and argument addresses out to the foreign thread and reading the (address of the) result back after the call) and registering the pipe/socket in the Gambit runtime, but using futexes instead of pipes *might* make it faster (but then maybe not); obviously such a layer would solve the problem of FFI calls blocking green threads. BTW iirc GHC(?) does implement this (I don't know details) in it's FFI (including a flag passed to each foreign function definition whether it's a quick or potentionally blocking call).
(PS. Brad, in case you didn't know already: gcc 4.2(.2+) also has "vector instructions through built-in functions" (access to MMX, 3Dnow! and SSE), which might possibly be interesting for you. But then, maybe not if your problems are memory bandwidth limited.)
From what I've seen, the experiences of the GHC implementors working on a parallel system (and at first, iirc, a concurrent GC?) may be worthwhile readings.
BTW the chicken people have also had some interesting discussion about GSoC participation in the last few days (don't be sad about the difference in volume), also including a few points about threading. Gambit does have a better chance of doing fancy stuff with garbage collection (because of it handling the continuation stack explicitely), of course.
- Gambit packages
Hm, should I try to offer mentoring: * working on chjmodule (renaming it at that occasion)? * working on some other module system?
I see that mentors must be listed already at the organization application. I'll release my current code quickly now (give me 24h). Sorry about being so late.
- web framework
Should I mentor on this? I've actually got some actual professional (well, that doesn't mean particularly successful) experience in this area (with an XML background). But I'd rather prefer having a module system done before continuing writing suboptimally modularized code as I did up to now.
- documentation
What comes to mind re documentation:
- make Gambit internals more approachable. (Documentation, compilation in safe mode, runtime reloading of separately compiled system parts.)
Other ideas:
- implement ability to 'migrate' continuations from compiled code to interpreted code. I.e. you would capture (and possibly serialize) a continuation of an error situation which happened in compiled code, store it away, then when you get to look at the problem, you load all code that you think is relevant for debugging into the *interpreter* (uncompiled) and then let the system turn that continuation into one that calls the interpreted procedures instead of the compiled ones. The idea is to be able to debug the problem much better (like use the stepper, change procedures on the fly etc.). How impossible would that be?..
- especially if the above won't work at all: change the interpreter so that it analyzes the lifetime of lexical variables and doesn't hold on to objects in variables longer than up to their last reference, to prevent the memory retention problems that otherwise plague lazy code. And possibly while doing so: try to reduce the size of serialized continuations from interpreted code.
- optional step-back debugging ("reverse in time"), possibly as global syntax transformation hook, by storing away the last n continuations seen by the interpreter (or also compiled programs; that's why implementing it as transformation might be preferrable).
- François Pinard's projects (see MSLUG mailing-list archive)
- regarding the Gambit<->Python interaction over a pipe/socket: having written Gambit->Perl bindings (although still missing callbacks from Perl into Scheme), I can say that linking two languages in the same process through the FFI is certainly possible. But it currently misses the above-mentioned FFI-in-a-separate-pthread approach to prevent Perl code from blocking Gambit green threads. Also running the other language in a separate process is safer against crashes, of course (although this hasn't been a problem for me so far). I did it in-process because I didn't want to risk performance problems. For example I don't normally convert a perl hashtable into a Gambit table when being returned from a call to perl code, but instead return a foreign object around the perl object, on which the Scheme code can then call perl's hashtable operators to get at the individual values. That saves the overhead of conversion of the whole table if you're only interested in a few values. The round trip times over a socket may make that relatively costly. (It will be better with Termite+multiple processes and a common shared read-only storage, since you can then read from the common storage and save the round trips :). I'll help doing some benchmark if you want to compare.
- speaking of pymacs: wouldn't it be nice to have a Elisp<->Gambit interface? (schemacs or something). But about the same thing might also be achievable through a SLIME48 ported to Gambit, which itself might be another good candidate for a GSoC project.
Christian.
Christian Jaeger wrote:
From what I've seen, the experiences of the GHC implementors working on a parallel system (and at first, iirc, a concurrent GC?) may be worthwhile readings.
How does Erlang handle this? The latest versions handle multicore/multiprocessor if I remember correctly.
BTW the chicken people have also had some interesting discussion about GSoC participation in the last few days (don't be sad about the difference in volume), also including a few points about threading. Gambit does have a better chance of doing fancy stuff with garbage collection (because of it handling the continuation stack explicitely), of course.
Is there some documentation about this somewhere? I'd love to read it.
- web framework
Should I mentor on this? I've actually got some actual professional (well, that doesn't mean particularly successful) experience in this area (with an XML background). But I'd rather prefer having a module system done before continuing writing suboptimally modularized code as I did up to now.
I'm not sure a "web framework" is that great an idea. "Frameworks" work best when they are scratching a particular itch. In addition, any good "web framework" for Scheme is likely to be not very Gambit specific.
-a
Andrew Lentvorski wrote:
Christian Jaeger wrote:
From what I've seen, the experiences of the GHC implementors working on a parallel system (and at first, iirc, a concurrent GC?) may be worthwhile readings.
How does Erlang handle this? The latest versions handle multicore/multiprocessor if I remember correctly.
Just to make clear about what we are talking: Erlang "processes" do not share state (in the form of mutations that could be directly visible in other processes), from the point of view of a user they are *not* shared memory threads. And, not only does the language not offer direct cross-process mutations, but their implementation traditionally did use a separate heaps for every process, even in their user-space scheduled implementation, so not even the *implementation* is/has been using shared memory (except probably for (immutable) program code). As long as they keep it that way, it's dead easy running multiple OS threads, you simply use the OS communication instruments for passing messages between the threads. Whether using OS threads is significantly faster than OS processes in this case is what I have been wondering about several times, numbers would be interesting here (I guess it would eliminate (or reduce) the costs of setting up MMU tables during context switches between OS threads of the same process, other than that, I see no reason for a speedup).
Now Scheme is obviously different in that it offers mutation and if you want to make the mutations been passed over to other processes (like the heavy math vector calculation people would like), the shared-nothing approach doesn't work. (I've already suggested using separate shared memory for such tasks.)
But on top of that, even if you're using Termite (which does not support mutation): even if you strip shared-memory mutation from Scheme completely (e.g. apply the Termite semantics to the underlying Scheme system), there is one thing left: Termite is so fast in passing big data structures between Termite "processes" (Scheme threads) because Termite "processes" are employing underlying shared memory storage. So no serialization/deserialization has to be done for local data transfers. Now you can't simply run multiple Termite "threads" (i.e. underlying Scheme threads) on different OS threads anymore, because you've got a heap that needs synchronization. You now need a concurrent GC etc., something like what non-green threaded Java virtual machines do (or something less scalable like the global interpreter lock of Python).
Now that being said, Erlang people have been discussing moving to using shared heaps, just like Termite. Now the interesting question is: have they managed to efficiently implements heaps being shared *across OS threads*? Note that if they did not (e.g. one heap is only ever green-threaded), they are simply using the Gambit approach, but saving the above mentioned MMU overhead. This would be the approach of running multiple Gambit interpreters in separate pthreads in the same OS process. This should be quite easy (but, as mentioned, I wonder how much of a difference that will make?). Do you have pointers to corresponding Erlang posts/docs?
BTW the chicken people have also had some interesting discussion about GSoC participation in the last few days (don't be sad about the difference in volume), also including a few points about threading. Gambit does have a better chance of doing fancy stuff with garbage collection (because of it handling the continuation stack explicitely), of course.
Is there some documentation about this somewhere? I'd love to read it.
(I've got that knowledge gradually over time from experiments / mailing list posts / reading the sources. I suggest to start by reading the paper Marc has posted on March 5th.)
- web framework
Should I mentor on this? I've actually got some actual professional (well, that doesn't mean particularly successful) experience in this area (with an XML background). But I'd rather prefer having a module system done before continuing writing suboptimally modularized code as I did up to now.
I'm not sure a "web framework" is that great an idea. "Frameworks" work best when they are scratching a particular itch. In addition, any good "web framework" for Scheme is likely to be not very Gambit specific.
Well, of course I've got some ideas on how that framework should look like.
It could be open to plug Termite in.
Of course you've got a point in that you could most probably port it to other Scheme systems, and the same of course applies to frameworks developed on other systems, they could be ported to Gambit. But if there is no Scheme system agnostic organization out there willing to take on the task of implementing such libraries, then this is being left to the individual system development organizations. I see no reason for Gambit playing purely a library consumer, as opposed to contributor as everyone else. Of course creating such a framework could also leverage (by porting) *parts* of other projects. Two things to keep in mind are: (a) system specifics to make something efficient; (b) module system specifics to make something particularly elegant. Part (a) could help show off Gambit's performance, of course (but yeah, is there a user out there right now who wants this?), part (b) is why I'd prefer seeing module stuff done first.
Christian.
I wrote:
Hm, should I try to offer mentoring:
- working on chjmodule (renaming it at that occasion)?
- working on some other module system?
I see that mentors must be listed already at the organization application. I'll release my current code quickly now (give me 24h).
Sorry, I can't make it so quick. Believe me, I really just started working again on it two days ago (it's only six days since I'm not tied up in a job anymore, too), and adapting to the newer Gambit releases is a little more complicated for me than I had wished; what I'm hitting against right now is the change in Gambit to not define standard forms like define or lambda in non-"" namespaces anymore; this is a sensible change, but I really have to build clean identifier handling in now, I don't just want to include ~~/lib/gambit#.scm because then one couldn't override any such identifiers anymore without globally affecting all code.
I'll continue hacking after this mail (I'll also analyze how Snow does things), expect me to invest half of my time of the upcoming days.
Let me advertise my planned work (in which a student could participate or take on parts) like this:
- study enough Gambit internals in the area of parsing/compilation stages, macro expander integration, repl handling, and debugger displays, symbol table handling, and possibly more, to be able to adapt and integrate any module system work cleanly.
- create a module system that's easy and practical for everyday work, meaning highly interactive, and at the same time aiming to enable optimizations which can lead to fast code without compromising code clarity (like e.g. cross module inlining, and specialization for parametrized modules).
- experiment also with the practical aspects on the filesystem side (filesystem layout, compilation caching strategies, C code compilation / some sort of make tools integration (integrate make tool calls into the module system and not vice versy!)).
- especially after the above infrastructure is well in place, study the existing approaches of modularization and syntax handling out there (Scheme48, MzScheme, Chicken's attempts at mixing syntax systems, riaxpander (http://mumble.net/~campbell/darcs/riaxpander/), R6RS, ERR5RS)
- think of every way that object oriented practices are being used to achieve modularization in OO languages and find the right counterpart for Scheme. (There will also probably always be *some* need for dynamic type dispatch (virtual method calls), be open to interface to a CLOS like OO system for this.)
- code is data, data is code: treat files that other languages would consider data (e.g. XSLT files) as modules. Also explore XML namespaceing ideas in s-expr syntax.
- those findings which are deemed important (and are not Gambit-specific, e.g. performance oriented), are meant to become portable by integration into or building on top of the Snow framework.
I would be happy if a student or other people would take part in this. So I would like to offer to be a mentor. I think we may not get done everything of the above during this summer, so if you want to participate (or also if you don't) help pick out what you think is important.
If you think the GSoC job description should be more precise than that, please tell me, so that we can come up with something better.
Christian.
Christian Jaeger christian@pflanze.mine.nu writes:
- study enough Gambit internals in the area of parsing/compilation
stages, macro expander integration, repl handling, and debugger displays, symbol table handling, and possibly more, to be able to adapt and integrate any module system work cleanly.
- create a module system that's easy and practical for everyday work,
meaning highly interactive, and at the same time aiming to enable optimizations which can lead to fast code without compromising code clarity (like e.g. cross module inlining, and specialization for parametrized modules).
I would be very interested in this kind of work, especially studying Gambit's internals. It would overlap some with my plans of prototyping a native threading strategy this summer.
- experiment also with the practical aspects on the filesystem side
(filesystem layout, compilation caching strategies, C code compilation / some sort of make tools integration (integrate make tool calls into the module system and not vice versy!)).
Essentially a build system. It would be nice to have a build system on top of a module system. (I don't think make tool calls should be integrated with modules, but rather placed on top of.)
I would be happy if a student or other people would take part in this. So I would like to offer to be a mentor. I think we may not get done everything of the above during this summer, so if you want to participate (or also if you don't) help pick out what you think is important.
Count me in.
James Long wrote:
I would be very interested in this kind of work, especially studying Gambit's internals. It would overlap some with my plans of prototyping a native threading strategy this summer.
Good, albeit I expect most things related to module handling to be in the early stages of compilation and in the user interface of the debugger, but broad knowledge of the internals will always be good.
- experiment also with the practical aspects on the filesystem side
(filesystem layout, compilation caching strategies, C code compilation / some sort of make tools integration (integrate make tool calls into the module system and not vice versy!)).
Essentially a build system. It would be nice to have a build system on top of a module system. (I don't think make tool calls should be integrated with modules, but rather placed on top of.)
I'm very open in this area. The reason why I've stated it the other way around is that the "normal" case, i.e. modules consisting simply of Scheme files that need no special compilation options, should not require any build setup, those should compile transparently when needed just like scripting languages like Perl load their files (if they are not containing foreign code, and note that the Perl guys went so far as to create the Inline::C stuff which removes the burden of dealing with make stuff again).
chjmodule currently supports cc-opts and ld-opts declarations which take a string to be fed to the compiler or linker for the cases where the Scheme file needs to be compiled/linked with special options; I'm open on whether to keep this or do something else. (Currently the module declaration forms are not being evaluated (they are not Scheme code), so there's no way creating the string depending on the host system etc., which is obviously wrong because it's not portable; making the module language (a subset of) executable Scheme would solve this; good idea or not?)
I would be happy if a student or other people would take part in this. So I would like to offer to be a mentor. I think we may not get done everything of the above during this summer, so if you want to participate (or also if you don't) help pick out what you think is important.
Count me in.
Cool!
Christian.
I wrote:
Cool!
James Long and I have met on the IRC and have now basically already started the project. We(*) have created a section on the wiki here:
http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Module_System
(* well, I have written the pages, James may not yet have verified them)
The system does not yet have a name because we want to wait with that until we know the team is complete.
James is not a student (anymore), and we think we are open for another core contributer, which still leaves up a possibility for one student.
We are currently wildly optimistic, and our project will get us enough building blocks in the way to make it fun and chances realistic that we hit a stumbling rock somewhere. We're out to learn the hard way all the non-working things that generations of scientists didn't have a chance of writing successful papers about.
I hope it will advance Gambit nonetheless, and I'll keep an eye on the whole thing to at least stay working in a good basic version for daily usage.
So, wish us good luck (and tell us when we go wrong)!
Christian.
Good luck! :)
On Thu, Mar 13, 2008 at 3:05 AM, Christian Jaeger christian@pflanze.mine.nu wrote:
James Long and I have met on the IRC and have now basically already started the project.
Cool.
From the wiki log-page:
"and have discussed why hygienic macro systems rename symbols and that when using namespaces in the symbols, this might be enough as "rename" means, thus providing nice integration with preexisting namespace-using Gambit code."
This seems interesting. Is there a chat-log, blog or paper describing this?
Also I kind of assume you have read or at least know of "Composable and Compilable Macros" by Flatt a good read when you put together a "syntactic tower".
On Mar 14, 2008, at 9:57 AM, Joel Borggrén-Franck wrote:
"and have discussed why hygienic macro systems rename symbols and that when using namespaces in the symbols, this might be enough as "rename" means, thus providing nice integration with preexisting namespace-using Gambit code."
This seems interesting. Is there a chat-log, blog or paper describing this?
There's a wealth of information on hygienic renaming. Two examples:
http://citeseer.ist.psu.edu/109510.html http://citeseer.ist.psu.edu/bawden88syntactic.html
As far as the renaming scheme goes, it seems somewhat arbitrary, as long as you're able to recover the correct environment from the renamed identifier (so Gambit namespaces might suffice, but there's also an issue with hygienic "security").
Our discussions on this are scattered but you're welcome to read through our brainstorms on the scheme channel (I have since done some research and understand things better).
http://ircbrowse.com/channel/scheme/20080312#hour20
Also I kind of assume you have read or at least know of "Composable and Compilable Macros" by Flatt a good read when you put together a "syntactic tower".
I have heard of it but have not read through it. I will look through it.
- James
On Fri, Mar 14, 2008 at 1:57 PM, Joel Borggrén-Franck < joel.borggren.franck@gmail.com> wrote:
"and have discussed why hygienic macro systems rename symbols and that when using namespaces in the symbols, this might be enough as "rename" means, thus providing nice integration with preexisting namespace-using Gambit code."
Well, that's what I'd call a viable engineering solution, but I doubt that it truly solves the problem. Macros can capture names even within a single namespace, can't they? However, by reducing the number of symbols in the namespace (since core Scheme has only one), you reduce the probability of capture - assuming that names are randomly distributed in the space of all possible identifiers.
Which of course they're not. But that doesn't mean that ny controlling the boundaries of the problem that it doesn't become easier to write real code. I think that long experience is showing that hygienic macros are great, but they don't really serve as a practical substitute for the ability to break hygiene when necessary. And frankly, good old define-macro is hard to beat for that, no matter how groovy syntax-case appears to everyone.
I'm not really sure why I responded here, maybe it's just the effects of 3 hours of sleep in the last 36 :)
Long live define-macro!
david rush
Date: Mon, 17 Mar 2008 20:33:15 +0000 From: "David Rush" kumoyuki@gmail.com
On Fri, Mar 14, 2008 at 1:57 PM, Joel Borggr�n-Franck < joel.borggren.franck@gmail.com> wrote:
"and have discussed why hygienic macro systems rename symbols and that when using namespaces in the symbols, this might be enough as "rename" means, thus providing nice integration with preexisting namespace-using Gambit code."
Well, that's what I'd call a viable engineering solution, but I doubt that it truly solves the problem.
With a suitable definition of `rename' it does.
Macros can capture names even within a single namespace, can't they? However, by reducing the number of symbols in the namespace (since core Scheme has only one), you reduce the probability of capture - assuming that names are randomly distributed in the space of all possible identifiers.
Which of course they're not. But that doesn't mean that ny controlling the boundaries of the problem that it doesn't become easier to write real code. I think that long experience is showing that hygienic macros are great, but they don't really serve as a practical substitute for the ability to break hygiene when necessary. And frankly, good old define-macro is hard to beat for that, no matter how groovy syntax-case appears to everyone.
This is absurd. The ability to break hygiene when necessary is offered by every reasonable programmatic macro system out there, not merely SYNTAX-CASE, yet long experience has shown that this is almost always a mistake; and DEFINE-MACRO is at best an excellent source of insanely obscure bugs that will turn up when you least expect, because a macro's description failed to include a subtle detail about what names it expects not to be rebound. Such obtrusive implementation detail is a necessary part of an unhygienic macro's interface, just like the names of local variables are in a dynamically scoped language; this is a very real problem in engineering large systems.
On Mon, Mar 17, 2008 at 8:43 PM, Taylor R Campbell campbell@mumble.net wrote:
hygiene when necessary. And frankly, good old define-macro is hard to beat for that, no matter how groovy syntax-case appears to everyone.
This is absurd.
It's a matter of emphasis, Taylor. I write better than 90% of my macros in SYNTAX-RULES. I *like* hygiene.
offered by every reasonable programmatic macro system out there, not merely SYNTAX-CASE,
SYNTAX-CASE and explicit renaming (Larceny & S48, IIRC) are the main ones I'm aware of. Are there other major contenders?
yet long experience has shown that this is almost always a mistake;
I never said anything different. But for the rare cases where hygiene breaking is desired, I find the semi-hygienic systems to be awfully cryptic.
And since I'm not interested in a religious advocacy war over hygiene, I'd like to know how you think module/namespace-based renaming is an adequate substitute for true hygiene in macros? Because I just don't see it. Module-based renaming and macrotic renaming operate at sufficiently different levels of detail as to seem like rather different operators to me.
david rush
Date: Mon, 17 Mar 2008 21:16:51 +0000 From: "David Rush" kumoyuki@gmail.com
SYNTAX-CASE and explicit renaming (Larceny & S48, IIRC) are the main ones I'm aware of. Are there other major contenders?
Explicit renaming is a special case of syntactic closures. I'm not sure whether Andre van Tonder's macro system (one of them high-numbered SRFIs) counts as another; in spirit it is explicit renaming with a dynamically scoped renaming procedure in the macro engine.
yet long experience has shown that this is almost always a mistake;
I never said anything different. But for the rare cases where hygiene breaking is desired, I find the semi-hygienic systems to be awfully cryptic.
In local macros that you plan to use in only one place to abbreviate the construction of elaborate names, it is not necessary to concern yourself with hygiene, and you can write a macro transformer procedure with explicit renaming or syntactic closures that looks identical to the DEFINE-MACRO version. But for anything beyond purely local macros, if you don't concern yourself with hygiene, however cryptic it may be, you are bound to lose later on.
Here is an exercise for the reader to illustrate the problem. Consider, for example, a DEFINE-ENUMERATION macro, defined so that
(define-enumeration foo (bar baz quux))
defines FOO to be a dispatching macro such that (FOO BAR) expands to 0, (FOO BAZ) to 1, and (FOO QUUX) to 2; and defines FOO-ENUMERANDS to be the vector #(BAR BAZ QUUX). Now suppose we write a SYNTAX-RULES macro that uses DEFINE-ENUMERATION:
(define-syntax mumble-frotz (syntax-rules () ((MUMBLE-FROTZ a b (c ...)) (BEGIN ... (DEFINE-ENUMERATION AN-ENUMERATION (c ...)) ... (FROB GROVEL AN-ENUMERATION-ENUMERANDS B))))).
Let's suppose DEFINE-ENUMERATION inserts the literal symbol AN-ENUMERATION-ENUMERANDS into the output:
(define-syntax define-enumeration (er-macro-transformer (lambda (form rename compare) ... (,(rename 'DEFINE) ,(string->symbol (string-concatenate (map symbol->string (list (name->symbol enumeration-name) '- 'ENUMERANDS)))) (,(rename 'QUOTE) ,(list->vector components))) ...))).
(We could even strip the renaming if you want, and make it look more like the analogous DEFINE-MACRO.) Why won't MUMBLE-FROTZ work? Why won't AN-ENUMERATION-ENUMERANDS in the output of MUMBLE-FROTZ refer to the variable we want?
And since I'm not interested in a religious advocacy war over hygiene, I'd like to know how you think module/namespace-based renaming is an adequate substitute for true hygiene in macros? Because I just don't see it.
I'm not interested in religious advocacy either; what I say here is a purely technical judgement.
Module-based renaming and macrotic renaming operate at sufficiently different levels of detail as to seem like rather different operators to me.
Hygienic renaming does not necessarily just mean affixing to each name a tilde or octothorpe and a big or random number that we hope is unique. Hygienic renaming means associating with each name its meaning, which may for local variables imply the affixing of funny-looking suffixes, but for names whose meaning must persist uniquely throughout a large system of components, it more likely implies associating some description of the module from which the name originated, or a path by which to get there. There is no fundamental, conceptual distinction between `module-based renaming' and `macrotic renaming'.
Take, for example, a DELAY macro, in some module that exports FORCE and DELAY and nothing else (perhaps throw EAGER and LAZY in to appease those of us who actually want to use laziness). We want to define the DELAY macro in terms of an internal MAKE-PROMISE:
(define-syntax delay (syntax-rules () ((DELAY expression) (MAKE-PROMISE (LAMBDA () expression))))).
But the relevant module does not export MAKE-PROMISE, so we can't just insert the literal symbol MAKE-PROMISE into the output, even if the user hasn't locally shadowed that name: if we insert the literal symbol in DELAY's output, it would be just as if the user had written MAKE-PROMISE in her own code, and she would encounter an undefined variable error.
Hygiene guarantees that the *name* (not `symbol') in the output of DELAY refers to exactly the same meaning as the symbol MAKE-PROMISE meant in the environment where the DELAY macro was defined. Exactly how this name is represented is inessential to the point of hygiene. It could be a qualified reference directly to the LAZINESS module, if we have a global naming scheme for modules. In Scheme48, it will be a `generated' name, which remembers: the name used to invoke the macro transformer that generated it, and the name by which the transformer identified it. Both names may be generated names themselves, but if we follow them all the way to the end, by recursively looking up the macro transformer, and looking up the name in that transformer's environment, we shall eventually end up with the original meaning we sought.
Whether this is the duty of the module system or of the macro system doesn't really matter -- that's an implementation detail, and requires only that the module system and the macro system cooperate. That it work is what matters.
Joel Borggrén-Franck wrote:
Good luck! :)
Thanks!
From the wiki log-page:
"and have discussed why hygienic macro systems rename symbols and that when using namespaces in the symbols, this might be enough as "rename" means, thus providing nice integration with preexisting namespace-using Gambit code."
This seems interesting. Is there a chat-log, blog or paper describing this?
As James has already answered, part of our IRC discussion is online.
Also I kind of assume you have read or at least know of "Composable and Compilable Macros" by Flatt a good read when you put together a "syntactic tower".
I haven't known it; thanks for the hint, I've started reading it yesterday but didn't get through yet.
BTW we have started a collection of relevant literature on the subject here: http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Module_System/Virtual_...
We are aware that there's much information out there; the point is not creating a comprehensive list here. We will list only what we have decided that we are going to read (or have read already). Of course your input will have an influence on what this will be.
Christian.
You may have noticed already that Gambit has not been accepted for the GSoC 2008 program, sadly.
What does this mean for our module system project?
We want to proceed as planned, even if, certainly, a part of the impetus got lost.
In the upcoming days, we're going to think about the inputs we've already got so far, and will report back again once we've got some news.
We think there will still be enough place for more volunteers. Just talk to us if you're interested (for example on #gambit on irc.freenode.net); or just send us your thoughts.
Thanks for caring & keep up with the interesting comments!
James & Christian
On Wed, 19 Mar 2008, Christian Jaeger wrote:
You may have noticed already that Gambit has not been accepted for the GSoC 2008 program, sadly.
What does this mean for our module system project?
We want to proceed as planned, even if, certainly, a part of the impetus got lost.
In the upcoming days, we're going to think about the inputs we've already got so far, and will report back again once we've got some news.
We think there will still be enough place for more volunteers. Just talk to us if you're interested (for example on #gambit on irc.freenode.net); or just send us your thoughts.
Thanks for caring & keep up with the interesting comments!
James & Christian
Perhaps talk with the Lispers of NYC:
oo--JS.
Date: Wed, 12 Mar 2008 14:52:40 +0100 From: Christian Jaeger christian@pflanze.mine.nu
- especially after the above infrastructure is well in place, study the existing approaches of modularization and syntax handling out there (Scheme48, MzScheme, Chicken's attempts at mixing syntax systems, riaxpander (http://mumble.net/~campbell/darcs/riaxpander/), R6RS, ERR5RS)
A word or two of advice on this subject. `Mixing syntax systems' in the manner pioneered by Chicken is an excellent way to lose badly, and it would be a mistake to replicate this lossage elsewhere. Also, unless you think about macros in conjunction with modules from the beginning, you will not have an easy time tacking them on later. Riaxpander makes parts of this easier, but you'll still need to think about these issues well in advance.
Taylor R Campbell wrote:
Date: Wed, 12 Mar 2008 14:52:40 +0100 From: Christian Jaeger christian@pflanze.mine.nu
- especially after the above infrastructure is well in place, study the
existing approaches of modularization and syntax handling out there (Scheme48, MzScheme, Chicken's attempts at mixing syntax systems, riaxpander (http://mumble.net/~campbell/darcs/riaxpander/), R6RS, ERR5RS)
A word or two of advice on this subject. `Mixing syntax systems' in the manner pioneered by Chicken is an excellent way to lose badly, and it would be a mistake to replicate this lossage elsewhere. Also, unless you think about macros in conjunction with modules from the beginning, you will not have an easy time tacking them on later. Riaxpander makes parts of this easier, but you'll still need to think about these issues well in advance.
Thanks for the heads-up, Riastradh.
Yes, thoughts about syntax issues will have to be part of the work as soon as possible. What I currently wonder about is whether a "syntactical tower" (if that's the right term for a way to process source code in an ordered pipeline of predefined processors, each accepting a certain language as input and delivering another ("lower level") one as output) can solve the problems.
BTW is there some documentation or discussion to read up about your work (objective, inner workings..) on Riaxpander somewhere?
Christian.
project suggestions, then please speak up on the mailing list.
There is a nice company, called LShift, that has developped Erlang servers and clients for the AMPQ protocol, their product being OpenSource and called "RabbitMQ".
AMQP stands for Advanced Message Queuing Protocol and is a nice protocol that defines a unified model for sharing data between applications. It's a typical middleware application, yet very powerful. It defines various "routing methods" based on the number of consumers and publishers and on possibly arbitrary complex fields in the message to enable STMP-like behaviour, as well as NNTP or real-time streaming. AMPQ is designed to handle critical data in critical domains (finance) with high responsivity.
Erlang is indeed a language that makes this protocol relatively easy to implement.
How about writing an implementation of AMQP with Termite? Success in this sector would definitely be a nice example of Scheme in the industry (with all your money relying on it ^^)
P!
How about writing an implementation of AMQP with Termite? Success in this sector would definitely be a nice example of Scheme in the industry (with all your money relying on it ^^)
Why try competing with an existing product, written in a language that greatly eases its implementation? Unless you can convince me that Termite could ease the implementation of really distinctive features that would be hard (if not impossible) to implement in Erlang, I'm not sure that's such a nice idea.
I think Scheme, as a language, is still in need of a niche market where it could really stand out and provide a competitive advantage. But don't get me wrong. I'm not talking about programmer productivity here. On this front, Scheme/Lisp is a definite killer. But you cannot take a large market share in any industry solely based on programmer productivity (*). That's not how business works, unfortunately for us... You need a kiiler app that can't be easily reproducible in most other languages (the barrier to entry).
(*) Maybe for businesses in the SaaS (Software as a Service) space, where the implementation language is totally irrelevant to the end-user, Scheme could help reduce development costs, provide higher margins and faster time to market (read: profitability!). It can even enable features difficult to replicate in other languages, thus providing a competitive advantage. Much like Lisp did for Viaweb in the 90's. But it must remain a secret weapon. I know at least one large company in the speech recognition industry that used Lisp before being acquired by Microsoft. But they were not advertising their use of Lisp at all.
Dominique
-
Adrien Pierard -
Andrew Lentvorski -
Christian Jaeger -
David Rush -
James Long -
James Long -
Jay Sulzberger -
Joel Borggrén-Franck -
Marc Feeley -
schemeway＠sympatico.ca -
Taylor R Campbell