On the #gambit IRC channel people were asking about the Gambit "roadmap".
Over the past months, the time I had to work on Gambit has been limited by another project, the implementation of a family of research VMs for JavaScript (interestingly a lot of the experience I have gained through Gambit is being applied to that project and there will be some returns to Gambit down the line). Our first paper has just been accepted to the Dynamic Language Symposium 2011.
Recently, I have been improving the Gambit REPL application for iPhone/iPad. The main new feature is the implementation of a public script repository, in fact the Gambit wiki, to upload and download scripts (the download requires a manual copy/paste operation to respect the Apple developer agreement).
I plan to resume work on the x86 back-end after that is done. My objective is to have a working system in 4 months. Performance of the generated code should be 2.5 to 5 times the performance of the C back-end on average, depending on the benchmark.
I will also resume work on supporting parallel execution. As a first step, the Gambit runtime will be refactored to allow multiple instantiations in the same OS process (to have multiple Gambit "VMs" run side-by-side, possibly communicating using sockets, or pipes, or some message-passing mechanism). That should not be too hard to do given the current structure of the runtime (each C module has a single "module" structure containing its global state).
Once that is done, the system will evolve to allow several OS threads to run in the same instance of the Gambit VM. For this to work, Gambit's current thread scheduler needs to be modified to multiplex the execution of the Scheme threads on multiple OS threads. In essence, the runtime system will start as many OS threads as there are processors (or cores) and the Scheme threads will be multiplexed on those OS threads. Gambit already manages stack allocation in a way that is easy to extend to multiple OS threads. Basically, each OS thread will allocate stack frames in a section that is specific to that OS thread. This eliminates contention for the stack and fragmentation of the stack memory. The Gambit synchronization primitives, such as mutexes and condition variables, need to be reimplemented to use the OS thread synchronization primitives. That is not conceptually hard, but it must be done very carefully otherwise there will be subtle bugs that could haunt the system for a long time. Another important issue is the GC. It too has to be made parallel otherwise the performance of parallel programs will suffer.
Thanks to a new research grant from Mozilla, I am reassigning some of my research money to work on Gambit. So if you are interested in Gambit and dynamic languages, and you are thinking of doing graduate studies (at the masters, PhD or postdoc level) on the implementation of dynamic languages, then please get in touch with me. I have openings!
Marc
Afficher les réponses par date
Hallo,
On Tue, Jul 19, 2011 at 11:01 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
On the #gambit IRC channel people were asking about the Gambit "roadmap".
This roadmap is simply awesome! I think we cannot thank you enough for all the high-quality effort you put on Gambit.
Thanks to a new research grant from Mozilla, I am reassigning some of my research money to work on Gambit. So if you are interested in Gambit and dynamic languages, and you are thinking of doing graduate studies (at the masters, PhD or postdoc level) on the implementation of dynamic languages, then please get in touch with me. I have openings!
I guess I live too far for graduating working on Gambit Scheme (that's a dream, for sure), but I would like to help with anything, if possible.
Cheers,
On Tue, Jul 19, 2011 at 05:01:09PM -0400, Marc Feeley wrote:
On the #gambit IRC channel people were asking about the Gambit "roadmap".
Over the past months, the time I had to work on Gambit has been limited by another project, the implementation of a family of research VMs for JavaScript (interestingly a lot of the experience I have gained through Gambit is being applied to that project and there will be some returns to Gambit down the line). Our first paper has just been accepted to the Dynamic Language Symposium 2011.
Recently, I have been improving the Gambit REPL application for iPhone/iPad. The main new feature is the implementation of a public script repository, in fact the Gambit wiki, to upload and download scripts (the download requires a manual copy/paste operation to respect the Apple developer agreement).
I plan to resume work on the x86 back-end after that is done. My objective is to have a working system in 4 months. Performance of the generated code should be 2.5 to 5 times the performance of the C back-end on average, depending on the benchmark.
I will also resume work on supporting parallel execution. As a first step, the Gambit runtime will be refactored to allow multiple instantiations in the same OS process (to have multiple Gambit "VMs" run side-by-side, possibly communicating using sockets, or pipes, or some message-passing mechanism). That should not be too hard to do given the current structure of the runtime (each C module has a single "module" structure containing its global state).
Once that is done, the system will evolve to allow several OS threads to run in the same instance of the Gambit VM. For this to work, Gambit's current thread scheduler needs to be modified to multiplex the execution of the Scheme threads on multiple OS threads. In essence, the runtime system will start as many OS threads as there are processors (or cores) and the Scheme threads will be multiplexed on those OS threads. Gambit already manages stack allocation in a way that is easy to extend to multiple OS threads. Basically, each OS thread will allocate stack frames in a section that is specific to that OS thread. This eliminates contention for the stack and fragmentation of the stack memory. The Gambit synchronization primitives, such as mutexes and condition variables, need to be reimplemented to use the OS thread synchronization primitives. That is not conceptually hard, but it must be done very carefully otherwise there will be subtle bugs that could haunt the system for a long time. Another important issue is the GC. It too has to be made parallel otherwise the performance of parallel programs will suffer.
The world has long needed a good, practical, flexible GC that can handle parallelism. The days that each programming language had to have its owm specialized and incompatible GC should be numbered. But running the GC in parallel with the program it's collecting is costly without hardware support.
The C-- project has some ideas how to manage the interface between a GC and its language. They may be right on how to do it, but I don't think their interfacing has yet been tested on anything parallel.
-- hendrik
Thanks to a new research grant from Mozilla, I am reassigning some of my research money to work on Gambit. So if you are interested in Gambit and dynamic languages, and you are thinking of doing graduate studies (at the masters, PhD or postdoc level) on the implementation of dynamic languages, then please get in touch with me. I have openings!
Marc
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On 2011-07-19, at 5:29 PM, Hendrik Boom wrote:
The world has long needed a good, practical, flexible GC that can handle parallelism. The days that each programming language had to have its owm specialized and incompatible GC should be numbered. But running the GC in parallel with the program it's collecting is costly without hardware support.
I'm not sure what you mean. Which hardware support? You mean special assembly instructions? That's doable in C with #ifdefs and inline "asm".
The C-- project has some ideas how to manage the interface between a GC and its language. They may be right on how to do it, but I don't think their interfacing has yet been tested on anything parallel.
I followed the C-- project way back when it started, and the last time I checked (2006) it hadn't lived up to all the hype. The project seems pretty dead at this point.
Marc
On Tue, Jul 19, 2011 at 08:54:12PM -0400, Marc Feeley wrote:
On 2011-07-19, at 5:29 PM, Hendrik Boom wrote:
The world has long needed a good, practical, flexible GC that can handle parallelism. The days that each programming language had to have its owm specialized and incompatible GC should be numbered. But running the GC in parallel with the program it's collecting is costly without hardware support.
I'm not sure what you mean. Which hardware support? You mean special assembly instructions? That's doable in C with #ifdefs and inline "asm".
There has to be coordination between the garbage-collector's accesses to memory and the program's modification of hte same memory. That involves significant overhead unless you can handle really fast the common cases where nothing much is going on very fast. I suspect that means manipulation of virtual memory or a variety of exotic indivisible instructons, but I haven't really looked at the details lately.
But I think that synchronising with the garbage collector is going to involve a significant performance hit, which might be offset by having lots of processors.
-- hendrik
The C-- project has some ideas how to manage the interface between a GC and its language. They may be right on how to do it, but I don't think their interfacing has yet been tested on anything parallel.
I followed the C-- project way back when it started, and the last time I checked (2006) it hadn't lived up to all the hype. The project seems pretty dead at this point.
I think it's been implemented on a few different processors now. And the language is a clear and elegant design. What it lacks is a lot of inplementation. That said, the design looks good. And none of the implementatinos do JIT compilation.
Well, the Hasell people say they've taken the basic abstract machine design and adapted and used it internally, using different data structures than files of lines of text. Of course, they've abandoned the notation, and their implementation is completely imdependent of the mainstream ones, so perhaps it was more a design motivation than anything else. Still, it may be worth looking at what they're doing. If it doesn't turn out that seeing what they've done is more work than doing it yourself. It does sometimes end up thhat way, doesn't it.
I don't know how C-- can implement C, though, with its vararg arguments.
-- hendrik _________________________________________________ For list-related administrative tasks: http://lists.racket-lang.org/listinfo/users
On 2011-07-19, at 9:47 PM, Hendrik Boom wrote:
I don't know how C-- can implement C, though, with its vararg arguments.
I think C-- can't implement Scheme's first-class continuations either, so that would be a problem for a Scheme compiler.
Marc
P.S. you sent your reply to the wrong mailing list.
On Tue, Jul 19, 2011 at 10:09:20PM -0400, Marc Feeley wrote:
On 2011-07-19, at 9:47 PM, Hendrik Boom wrote:
I don't know how C-- can implement C, though, with its vararg arguments.
I think C-- can't implement Scheme's first-class continuations either, so that would be a problem for a Scheme compiler.
Yeah, it could be. I'll have to check into that. It certainly does implement tail-calling properly, which lots of systems don't. But that's not the same thing at all.
Marc
P.S. you sent your reply to the wrong mailing list.
So I did! I resent it to the right one just now, so your list will have a complete record.
-- hendrik
On Tue, Jul 19, 2011 at 4:29 PM, Hendrik Boom hendrik@topoi.pooq.comwrote: The C-- project has some ideas how to manage the interface between a GC and its language. They may be right on how to do it, but I don't think their interfacing has yet been tested on anything parallel.
-- hendrik
This is an interesting comment. I concur with Marc that C-- looks to be dormant, especially now that Haskell has its LLVM backend. But I'm not particularly impressed with how LLVM handles garbage collection. If you don't mind my asking, what specifically about C--'s approach to GC/language interface did you find particularly compelling?
best, Jason
On Tue, Jul 19, 2011 at 08:04:36PM -0500, Jason E. Aten wrote:
On Tue, Jul 19, 2011 at 4:29 PM, Hendrik Boom hendrik@topoi.pooq.comwrote: The C-- project has some ideas how to manage the interface between a GC and its language. They may be right on how to do it, but I don't think their interfacing has yet been tested on anything parallel.
-- hendrik
This is an interesting comment. I concur with Marc that C-- looks to be dormant, especially now that Haskell has its LLVM backend.
I didn't know that Haskell had gone to LLVM.
But I'm not particularly impressed with how LLVM handles garbage collection. If you don't mind my asking, what specifically about C--'s approach to GC/language interface did you find particularly compelling?
MAybe the fact that they managed to integrate a mostly untyped langauge with a garbage collector that needs to knoww types. That they actually addressed the issue in their language design instead of ot looking like a total afterthought. And that they expect the C-- code generator to keep track of all the transient values in registers or wherever so that the garbage collector can actually get the information it needs to do its job. That, and that they've provided an interface to the write barrier that modern concurrent garbage collectors need to get any kind of speed. This stuff was designed in.
So the garbage collector can get, say, pointers to type-descritprs for the stuff on the stack. But as to what the type descriptors actually are, well, that's a matter between the high-level language and the garbage collector. C-- doesn't involve itself with it. I'm not sure whether this is a pity or not. A lot of the really tricky aspects of garbage collection don't care much about the details of the language. It would be useful to have that part available, only to be paranmetrized with the methods for interpreting type descriptors. It would be useful for that part, as a result, to be able to deal with the specific data layouts of different languages when they're used together in one program.
-- hendrik
-
Alex Queiroz -
Hendrik Boom -
Jason E. Aten -
Marc Feeley