On 14 Jun 2011, at 11:06, Per Eckerdal wrote:
I believe there are several key parts to the "batteries included" feel of a language ecosystem:
One of them is that it has to be easy to create a library and share it with others; otherwise a sufficiently large body of libraries won't be created. This has been the main motivation behind Black Hole: It could make it feasible to write and share libraries with non-trivial dependency graphs.
But I have realized that simply making it possible to share code easily isn't sufficient. Some important issues here are
- R5RS, being a minimal language, simply doesn't provide enough syntax (ie macros). The special forms are so few that each developer has to choose between using vanilla R5RS, which is often painful (the syntax for multiple values for instance), or creating his own set of base macros, which leads to loads of separate and suboptimal designs of things that should be core language constructs.
This works well for isolated projects with few dependencies to other libraries, but it seriously hurts code readability when sharing libraries.
- R5RS has a culture of not incorporating things into the language until it is known that the particular design is arguably the best possible. This works well when it is acceptable to have many different approaches to the same problems until they're solved, but it seriously hurts the ability to share code.
If one wants to make a "batteries included" system, this design philosophy simply doesn't work. It is necessary to incorporate things before they are perfectly worked out, and then improve things incrementally. (Of course this doesn't make careful design less important.)
The minimalistic, don't-add-until-perfect, way of things has left R5RS with several important (for projects that need "batteries included") holes in its core functionality, for instance a module system, unwind-protect, complete I/O, threads and even simple things like a basic set of list functions (fold, where are you?).
- Most "batteries included" languages nowadays come with a relatively large standard library, and rarely encourage multiple language implementations like Scheme does. I think discouraging multiple implementations of the language is an important ingredient to make a batteries included language work.
There can be more than one language implementation, but there has to exist an obvious default. Ruby, Python, Java, PHP, .NET, and to some extent C/C++/ObjC all do this.
Speaking as a scheme outsider until recently (so my view is probably not worth much), I think a big issue with Scheme is that the community has too many heros. Plenty of people are building clever implementations (my hat comes off and my thanks go out to them), but there seem to be relatively few undertaking the difficult but probably less glamorous and less instantly gratifying task, of making useful software development features accessible across as many implementations as possible. I'm not sure multiple implementations are a bad thing per se, but they do surely make it hard for a system of cross Scheme tooling to gain traction.
Doesn't a module system address your points 1, 2 & 3? –
1) Additional special forms living in modules could "compete" and grow. Perhaps one would finally become a de facto standard and be ratified by RnRS (or the best features of several libraries could form the basis of ratification, spinning off in to a new library that existing client libraries would develop towards). If RnRS doesn't want to include unnecessary details, it can just ignore this work, or an additional standards "track" could take it on.
2) Modules allow library and syntax to be developed without being formally included in the standard until they are ready. It seems to me this would complement the understandably gradual development of the language core very well.
3) A module system that can be hooked on to by existing Schemes (with some work on their part) allows them to develop in relative isolation, while the shared community also grows around the modules which act as a (de facto) standard library.
- I have argued that making it easy to share code isn't enough to give the language a "batteries included" feel: To get there, we'd have to change the whole language philosophy to the extent that we'd have a new language, complete with its own syntax, semantics, feel and culture.
Here I disagree, but as I said, I'm a newcomer.
Unification work isn't glamourous or attractive (or perhaps even a significant priority) for most of the people able to do it. It also probably has a significant "political" facet, rather than being purely technical. If I am right, this is a different from the underlying philosophy – it's more about people as a whole, rather than Scheme :-)
The existing syntax, semantics and feel are all pretty great from my point of view. They just have a few areas where more is needed. A widely supported module system should let this develop.
Yet I think it's important to remember that, for some applications, Scheme's simplicity and smallness are really nice. Examples of this are education and embedded apps.
The people who currently use Scheme now probably use it because it actually fits what they are trying to do. Changing the language this much would change the target audience of the language so much that I can't see how Scheme could evolve into a batteries included language.
As I've said above, I don't think such a big shift is needed. Assuming this is the case, the existing audience would be greatly augmented, rather than replaced. Existing implementers wouldn't need to hook their Scheme to a new module system. They could use the module system externally to pull down code (rather than having it available natively from within the language), or they could simply ignore it completely.
Racket matches what I have described above fairly well, but it has some design choices that I haven't (yet?) been able to come to terms with.
Jazz also seems to do this, and is derived from Gambit (I think), but I lost patience when I found I'd need to instal a pile of extra packages to get it working (which is certainly stupid of me and I should look again).
I am investigating the possibility to create my own Lisp based on these ideas, but I don't know yet if it would just hurt things by segmenting the Lisp world even further, or could help by contributing some new ideas and thoughts. I think that I at least know enough about Scheme and Lisp to be able to avoid repeating most historical mistakes though.
I don't think you'll hurt things, and perhaps you'll get the necessary traction for your implementation. Wouldn't it be easier (though much less exciting) to take an existing Scheme and patch it to support a minimal interface through which is can connect to an independently developing set of tools?
That said, I'm aware that to my mind, Black Hole with Gambit already seems to provide most of what would seem necessary to me. This must mean I've hopelessly misjudged the situation because were I correct, I think this would be evident to you too, and you would only consider your proposed course of action if you were an incorrigible glutton for punishment :-)
Thanks, Benjohn