I've had a draft e-mail sitting in an IMAP folder for about 8 months with the subject line "The future of Gambit", but with an empty body. Today I still don't know precisely what to put into this body, but at least I found a blog entry that explains a bit of my feelings.
I'm not exactly thrilled about the level of "outside" participation in (core) Gambit development. In fact, graphing the Gambit git commit log would be a parody of the argument in
http://www.gnome.org/~michael/blog/ooo-commit-stats-2008.html
that the lack of non-Sun developers working on OpenOffice.org is an indicator that it is not a healthy project. For there is only one committer to Gambit, and the best that can be said about getting other developers to develop code for Gambit is that a very few times Marc has committed code he hasn't written himself.
I've studied the Gambit source code a bit, and even made some contributions over the years, but Marc's programming style is something I have not seen elsewhere---highly macroized and layered, there are patterns in the coding techniques but not ones that I've been able to unravel in general. Perhaps one way to help new developers get into Gambit would be fore Marc to take a subsystem and write down an explanation of how the macros and layers of code for types, exceptions, constructors, functions, ... of that subsystem work. (Christian Jaeger seems to have had some success in doing this, but it would be good to have an explanation of how Marc sees the construction of subsystems of the runtime.) Just to get an idea of Marc's programming style would allow others to divine more easily the structure of the code.
And happy holidays to everyone!
Brad
Afficher les réponses par date
Bradley Lucier wrote:
I've had a draft e-mail sitting in an IMAP folder for about 8 months with the subject line "The future of Gambit", but with an empty body. Today I still don't know precisely what to put into this body, but at least I found a blog entry that explains a bit of my feelings.
I'm not exactly thrilled about the level of "outside" participation in (core) Gambit development. In fact, graphing the Gambit git commit log would be a parody of the argument in
http://www.gnome.org/~michael/blog/ooo-commit-stats-2008.html
that the lack of non-Sun developers working on OpenOffice.org is an indicator that it is not a healthy project.
I'm not sure you can draw direct conclusions from observations in broadly developed projects. I've come from using language implementations being developed by like hundreds of participants to Gambit and see it a more fertile ground for my work than any of those others.
I wouldn't go as far as saying that too many cooks always spoil the meal, but at the same time that some projects are having many contributors, they also don't offer a lean, understandable nor bugfree base either. Like, I'm understanding *much* more of Gambit's internals than I do about Perl's, even if am [have been] using Perl much longer than Gambit.
That said, I'd agree that a linear development history with commits by only one person looks funny in the context of a distributed version control system; but there may be a few points to say about this:
- many people may not yet know well enough how Git works; I'm happy about the move to Git, especially for the git format-patch feature, which retains user and commit info (and makes discussion easy), but so far I've been the only one sending such patches to the list
- I've got one patch (the one about the .gambc_history permissions) which has not been applied to the Gambit repo yet, and I don't know why, but it doesn't bother me much (except for having to not forget about it) -- what I can do is keep such patches in my personal branch, and if I feel that this branch is of interest to others I can publish it (either on repo.or.cz by using it's forking feature to clone the Gambit mirror there, or by asking Marc to get me a branch or a couple on his server; I'm not sure yet what's the better idea, I will discuss with him as soon as I get to it). (Then merging can start to happen and the history will stop looking linear.)
- I can observe that some of the heavy Gambit users are just happy asking Marc when they need a feature, and so far he seems to have been able to satisfy their needs; of course this way those users are getting higher quality results faster than if they'd have to do it themselves. Maybe we should find people (I'm hoping I won't be the only one) who can take part of this job instead, so that Marc will have more time for other new developments; but I'll see Marc within a few weeks time and will be able to discuss with him how work sharing might work out best.
I've studied the Gambit source code a bit, and even made some contributions over the years, but Marc's programming style is something I have not seen elsewhere---highly macroized and layered, there are patterns in the coding techniques but not ones that I've been able to unravel in general. Perhaps one way to help new developers get into Gambit would be fore Marc to take a subsystem and write down an explanation of how the macros and layers of code for types, exceptions, constructors, functions, ... of that subsystem work.
I don't expect good contributors will automatically come if one provides such info. But maybe you should point out the problems *you* are having understanding the code for what you want to do (and/or what you want to do), and then we'll answer each other's questions on the list, and maybe write some wiki pages as the result? Also note that quite some discussion during this year has taken place on IRC (on #gambit on freenode.net, and I've also been queried quite heavily by some people who preferred private chats); not much of it has made it to the wiki yet though.
(Christian Jaeger seems to have had some success in doing this, but it would be good to have an explanation of how Marc sees the construction of subsystems of the runtime.) Just to get an idea of Marc's programming style would allow others to divine more easily the structure of the code.
A few things:
- most macros are accessors from type definitions; if you know which types there are, you can understand many of those macros.
- I did complain to Marc about those macros quite some time ago, mainly because it prevented me from accessing structures at runtime, but (maybe as a result of the complaint?) now you can just (include "~~lib/_gambit#.scm") to get them back at runtime. (And the fact that just loading this file takes up a considerable amount of memory makes for a good explanation why those accessors are macros in the first place :).)
- you may want to take a look at my object-to-sexpr code if you want to quickly find out what a data structure contains:
git clone http://scheme.ch/gambit/experimental/object-to-sexpr/.git
(There *might* be a bug in it currently, I've seen cases where it shows part of it's own result as part of the result, but I can't explain that from my code, I'll have to discuss this with Marc some time.)
- Dpeschel has done some work to index Gambit sources, see http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/User:Dpeschel
- the JazzScheme IDE has nice cross indexing features; maybe I'll [help] look into how those can be brought to Gambit source code (non-Jazz Scheme code).
And happy holidays to everyone!
Thanks, a happy new year to everyone and the project
Christian.
Brad, as you know writing good documentation is not a simple matter... it takes time and considerable effort (at least for me) to write something that is correct and readable. My experience is that few people actually read documentation anymore, so the incentive is low. There are still some important features of Gambit that are not documented in the user manual. The reason is simply that the time I can devote to Gambit is mostly spent adding features that users request, fixing bugs, evolving the system, simplifying distribution, setting up and maintaining the wiki, mailing list and source code repository, etc. Internal documentation is not on the critical path!
I agree that it would help if more people were involved in sharing this workload. I do not believe the best place for (most) people to contribute is modifying the source code. I am ready to put more work into documenting the code and its organisation to make it easier to contribute to the code, but the time I can put on this will greatly increase if people can also take on some other tasks.
One thing that might also help is to have one or more experimental branches on the source code repository in addition to the master branch. The experimental branches would be under the control of individuals or groups of developers with a specific interest. When a feature has matured on an experimental branch it could be merged with the master branch.
Marc
On 28-Dec-08, at 9:16 PM, Bradley Lucier wrote:
I've had a draft e-mail sitting in an IMAP folder for about 8 months with the subject line "The future of Gambit", but with an empty body. Today I still don't know precisely what to put into this body, but at least I found a blog entry that explains a bit of my feelings.
I'm not exactly thrilled about the level of "outside" participation in (core) Gambit development. In fact, graphing the Gambit git commit log would be a parody of the argument in
http://www.gnome.org/~michael/blog/ooo-commit-stats-2008.html
that the lack of non-Sun developers working on OpenOffice.org is an indicator that it is not a healthy project. For there is only one committer to Gambit, and the best that can be said about getting other developers to develop code for Gambit is that a very few times Marc has committed code he hasn't written himself.
I've studied the Gambit source code a bit, and even made some contributions over the years, but Marc's programming style is something I have not seen elsewhere---highly macroized and layered, there are patterns in the coding techniques but not ones that I've been able to unravel in general. Perhaps one way to help new developers get into Gambit would be fore Marc to take a subsystem and write down an explanation of how the macros and layers of code for types, exceptions, constructors, functions, ... of that subsystem work. (Christian Jaeger seems to have had some success in doing this, but it would be good to have an explanation of how Marc sees the construction of subsystems of the runtime.) Just to get an idea of Marc's programming style would allow others to divine more easily the structure of the code.
And happy holidays to everyone!
Brad _______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On Jan 14, 2009, at 10:27 AM, Marc Feeley wrote:
I've studied the Gambit source code a bit, and even made some contributions over the years, but Marc's programming style is something I have not seen elsewhere---highly macroized and layered, there are patterns in the coding techniques but not ones that I've been able to unravel in general. Perhaps one way to help new developers get into Gambit would be fore Marc to take a subsystem and write down an explanation of how the macros and layers of code for types, exceptions, constructors, functions, ... of that subsystem work. (Christian Jaeger seems to have had some success in doing this, but it would be good to have an explanation of how Marc sees the construction of subsystems of the runtime.) Just to get an idea of Marc's programming style would allow others to divine more easily the structure of the code.
Marc:
Thank you for your reply.
I'm thinking something along the lines of Shriram's talk "The Swine before Perl", which goes through an example of how to program finite- state automata using define-syntax macros. To object to no documentation is not to demand full documentation---I've included one of my paragraphs above which explains that it would be nice to see how you think of programming. Pick a subsystem of the runtime or of the compiler (the register allocator, beta-reduction, lambda-lifting, one of them), explain what the layers are, how and why you do it. People would learn from it (I think I would be a better programmer if I understood it).
Brad
I've also wanted to fix a bug myself in Gambit several times but found the code to be a bit too mysterious. I agree that if one subsystem's coding convention's could be fully explained, this might crack the whole thing open. I know writing good docs is hard, perhaps it would be easier to convince *someone* to get up and start speaking in front of a camera and then just put it online. I think most people who are interested in getting some work done on the gambit internals really just want to know 'the big picture', namely.. what are the subsystems, how do they fit together, and what are the common coding techniques that make this happen.
On Wed, Jan 14, 2009 at 10:41 AM, Bradley Lucier lucier@math.purdue.edu wrote:
On Jan 14, 2009, at 10:27 AM, Marc Feeley wrote:
I've studied the Gambit source code a bit, and even made some contributions over the years, but Marc's programming style is something I have not seen elsewhere---highly macroized and layered, there are patterns in the coding techniques but not ones that I've been able to unravel in general. Perhaps one way to help new developers get into Gambit would be fore Marc to take a subsystem and write down an explanation of how the macros and layers of code for types, exceptions, constructors, functions, ... of that subsystem work. (Christian Jaeger seems to have had some success in doing this, but it would be good to have an explanation of how Marc sees the construction of subsystems of the runtime.) Just to get an idea of Marc's programming style would allow others to divine more easily the structure of the code.
Marc:
Thank you for your reply.
I'm thinking something along the lines of Shriram's talk "The Swine before Perl", which goes through an example of how to program finite- state automata using define-syntax macros. To object to no documentation is not to demand full documentation---I've included one of my paragraphs above which explains that it would be nice to see how you think of programming. Pick a subsystem of the runtime or of the compiler (the register allocator, beta-reduction, lambda-lifting, one of them), explain what the layers are, how and why you do it. People would learn from it (I think I would be a better programmer if I understood it).
Brad _______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
2009/1/14 Marc Feeley feeley@iro.umontreal.ca:
One thing that might also help is to have one or more experimental branches on the source code repository in addition to the master branch. The experimental branches would be under the control of individuals or groups of developers with a specific interest. When a feature has matured on an experimental branch it could be merged with the master branch.
Pardon me if I am being thick here, but isn't that the point of using a distributed SCM like Git or (as formerly) Mercurial? Every checkout is a branch...
david
David Rush wrote:
2009/1/14 Marc Feeley feeley@iro.umontreal.ca:
One thing that might also help is to have one or more experimental branches on the source code repository in addition to the master branch. The experimental branches would be under the control of individuals or groups of developers with a specific interest. When a feature has matured on an experimental branch it could be merged with the master branch.
Pardon me if I am being thick here, but isn't that the point of using a distributed SCM like Git or (as formerly) Mercurial? Every checkout is a branch...
I think the point is that Marc is encouraging others to go ahead and work on and publish their own branches.
Of course if nobody else knows enough about the internals to do any work, this won't happen, that's the point Brad is making.
I think we're aware of this situation. Some development is happening covertly, maybe for historic, maybe for cultural (academic environment) reasons, maybe it would be good to make this more open so that others are getting more chance for learning; but as I mentioned previously, systems which are being developped fully openly may gain much momentum but need not become more useful in the end. So I'm going to shut up and contribute.
Christian.
Marc Feeley wrote:
Brad, as you know writing good documentation is not a simple matter... it takes time and considerable effort (at least for me) to write something that is correct and readable. My experience is that few people actually read documentation anymore, so the incentive is low.
This is true, but probably not for the reason most people think.
What I do read is *example code*. A chunk of code that uses feature X to do some useful task Y is better than just about any amount of API documentation ever produced.
Getting people to push more code to the dumping grounds is probably vastly more useful than the API docs.
-a
Andrew Lentvorski wrote:
Marc Feeley wrote:
Brad, as you know writing good documentation is not a simple matter... it takes time and considerable effort (at least for me) to write something that is correct and readable. My experience is that few people actually read documentation anymore, so the incentive is low.
This is true, but probably not for the reason most people think.
What I do read is *example code*. A chunk of code that uses feature X to do some useful task Y is better than just about any amount of API documentation ever produced.
Getting people to push more code to the dumping grounds is probably vastly more useful than the API docs.
I think suggesting that examples are more useful than docs is a bit disproportionate; I did read more docs than other people's code to get my work done during the last ten years anyway. (And I think I've spent more time reading the source of the respective system/library I was using than reading code from other users.)
That's not to say that the dumping ground couldn't be useful, but I think what it is missing currently is active feedback; to make this easier, I think it would help much if all code there would be offered in a VCS. Not only makes this contributing changes back to the author easier, but it also allows for the readers of the dumping ground to actually see the feedback that happened. Offering VCS access is easy enough (tar including .git, or better pointing to an online repo; check out repo.or.cz, also offering a git repo over http on any http server you've got write access to is easy). Does anyone think easy Git hosting on the Gambit site (or even somehow making dumping ground offerings being checked in automatically) would be worthwhile?
Christian.
Bradley Lucier wrote:
I've studied the Gambit source code a bit, and even made some contributions over the years, but Marc's programming style is something I have not seen elsewhere---highly macroized and layered, there are patterns in the coding techniques but not ones that I've been able to unravel in general. Perhaps one way to help new developers get into Gambit would be fore Marc to take a subsystem and write down an explanation of how the macros and layers of code for types, exceptions, constructors, functions, ... of that subsystem work.
Speaking as someone who has ported Gambit once and simply couldn't port it a second time, the sheer amount of indirection is incredibly painful.
The thing is ... I think it's a *scheme* thing, or, perhaps more accurately, a Scheme-implemented-in-C thing.
If you dig under the covers of any of the actually useful schemes, they *all* have these painful levels of indirection. If you try to implement a scheme, you wind up creating similar levels of indirection the moment you hit the hard features like continuations, tail call optimizations, GC, etc.
I tried looking at various schemes for porting to embedded hardware. I was looking for something with a useful core that is around 1-2 KLOC. That's what all the "Schemeophiles" claim it takes to implement a Scheme.
They lie. It simply doesn't exist. Everything is 10 KLOC (and, really, that's so useless that you really need about 50-60 KLOC) and up and they all have incredibly painful amounts of indirection.
Sure, there are lots of small "schemes". Either they are metacircular and leave the tough stuff to an underlying scheme (not useful, thanks for playing), or they always have something that doesn't work: tail-calls are broken, continuations don't work, the macro systems are hosed, etc.
And, one of the things which I think contributes is the fact that the advanced features make other features more indirect. For example, tokenizing and parsing a Scheme expression is quite obnoxious using C. However, once you get continuations running, it is a big win to pull that code up from C into Scheme itself, pass the result back and hold the partial parse in a continuation. Of course, now the parser has this particularly opaque indirection in the middle of it that makes it hard for newcomers, but allowed the original maintainer to wipe out several hundred lines (at least) of C parsing code that really didn't work for a lot of corner cases anyhow.
Any scheme that becomes useful seems to be be very large and accumulates lots of indirections in the implementation.
-a
-
Andrew Lentvorski -
Bradley Lucier -
Christian Jaeger -
David Rush -
Marc Feeley -
|/|/ Bendick