Per Eckerdal wrote:
Ok, this discussion doesn't feel extremely urgent, but it's not that hard, and it has to be done some time or later, so let's begin.
(As I said I can't dig into the topic deeper right now, so this is probably my last mail until february.)
When I think about it, it is not that hard to interface to "legacy" code, using the namespace system. The module system actually uses it internally. Only partially, and I'm hoping to remove it relatively soon, because as I said, it's not nice to work with, but the naming convention namespace#identifier is used and I don't see why that would change.
You haven't said what isn't nice.
Using namespace prefixes for the automatic naming of identifiers deems me a good idea for these reasons:
- for debugging, you need some identifier syntax; going to some extent to keep those deterministic should make it easier for the user to type them in at the repl (i.e. something like foo:macrocall-bar#a would be deterministic, while a7481 isn't). - also for work on making different module systems interoperate it will be easier to have such readable representations.
You may have meant these when you said "debugging".
The module system (ehhh... we ought to find a good name for it)
If you'd like to have a name for what you have written, and intend to keep working (because you've got code written using it), now, then best choose a name yourself; could just be pe-modulesystem or so. I think we're going to be a number of people working on partly common, partly differing ideas for some time, so there's not going to be a single unified module system soon and I think it's best if each of the respective original authors keep responsibility for their code base and hence also for the name. Anyway, I can't tell before I've taken a deeper look.
is built on top of the notion of packages, which is a rather powerful mechanism. It should be possible to write code for a package that uses namespace. (Package is the word I use for a collection of modules) The problems I see are these:
Other people might be thinking differently than me here, but I really think that namespaces shouldn't be picked by the user.
- code always needs to specify a dependency somehow; so you will always have the problem of unique global naming in some way or another. - the idea to group together modules into packages, so that you can locally name the other modules and hence can use simple naming, looks sensible or at least interesting (although global naming for remote dependencies is not solved by this, and making two ways of dependency selection maybe adds some additional complexity). - systems which create code need to choose namespaces automatically, sure (for example for parametrization)
So, - a module developer will always have to pick *some* sort of *name*. (I did write you some suggestions in a private mail from 09/22/2008 to achieve this; may I quote that here?) - with plain old manual ##namespace handling, usually the namespace and the files (which are kind of the dependency name) are usually the same. - as mentioned above, for debugging (both system interoperation and at the repl) it would be nice to keep module name and namespace "close", like, module named "foo" in your module system might get namespace "pe-modsys:foo#".
I'm also thinking that a function in Gambit to list all identifiers in a particular namespace would be good to have (easy to write as a filter on all symbols, anyway; maybe it should be made scalable by using trees though).
The way to distinguish modules ought to be a more flexible one. It is really the same problem as prefixing in C, except that in this case there is a really clean solution IMO:
Because namespaces aren't explicitly shown to the user (except when debugging), the module system can choose a namespace for the module when compiling. (There must be a machine-global dictionary of taken namespace names or equivalent to ensure consistency.) This means I can make a CGI module, call it cgi.scm and release it. Someone else might make a different CGI module and call it cgi.scm. With this system, the modules can coexist, the first one installed will get the namespace cgi#, the second one might get cgi-2#.
To me, this is an important feature, both because it hides needless complexity from the user (choosing a namespace, which is redundant work to choosing the file's physical location)
As you've noticed already we're in agreement: I've never implied that you would use ##namespace directives in modules; chjmodule didn't, jazz doesn't, your system doesn't. But it's the suggested mechanism to keep to (a) make module systems work together and debug, (b) there is "legacy" code which would be nice to keep that way (it's like writing assembly, going to a lower level manually, and unsafe if you want to, might disappear with time, but useful for now).
and it doesn't require ad-hoc prefixes. The current "solution" seems to be that I ought to call the file something like pe-cgi.scm. To me, this solution sucks, because to be able to make complex software, you need to use hundreds of modules and organise them in a tree structure and I don't see how that fits neatly together.
What do you mean with "tree"? Filesystem tree? Would you take the filesystem tree 1:1 for the naming in dependencies? Then you still got the problem of conflicts, it's just going from pe-cgi to pe/cgi. (But you might have that "local naming" advantage as above--do you mean this? Are packages living in a "subfolder" for you? Or are they more akin to lexical scoping, where the outside world can't see the inside names?)
There are more reasons to why the module system should generate namespaces, mostly related to what a future ruby gems-equivalent tool would benefit from.
You should tell more.
Anyways. The problem I see is simply this: All "normal" modules will
(there is no "normal" for now--or, there are several definitions of "normal" for now)
have a machine-generated namespace, to avoid name clashes. This impossible to do with the old system, so namespace clashes might happen.
(Again, you didn't say how you avoid *name* clashes; but as mentioned, agreed on that the system chooses the namespace.)
The second problem I see is the one with dependency tracking. The system is based on the fact that it is able to track and automatically load dependencies. I don't see how that is done with namespaces and the #.scm convention.
(I don't think code using ##namespace manually will ever want to load a module of a module system; only the inverse. I.e. code using ##namespace will be at the leaves of the dependency tree. Usually anyway, maybe it's useful for hacking to be able to call module system loader functions from that lower level.)
Christian.