Hello
So, as I've discussed on IRC, I want to get back into module system
development. And as mentioned, I want to concentrate on "from-scratch
matters" for now to help make existing module system development
efforts' interoperation and further development possible.
What are my aims?
- having a base on which I can write own modularization approaches if
I'd like to.
- being able to use code written for other module systems.
- study how that interoperation between the various approaches can be
made to work.
- work together with all other interested parties; "as I see fit", at
least. The only thing I can say is that I'll try to avoid to tie
myself too much into particular usage perspectives; for me Scheme
(or a subset thereof, we'll see) is a means to layer languages on,
and I want to make this layering as easily workable as possible.
And I'm certainly going to focus development on Gambit as the
underlying system, for the foreseeable future.
(What we've once written in the wiki is still true for me:
http://dynamo.iro.umontreal.ca/~gambit/wiki/index.php/Module_System )
So what do I plan to work on at first?
I think that the most important thing right now is more
flexibility/power in the compilation of code pieces. What we have now
is |compile-file| (and of course batch compilation) and |load|, for
compilation; and we have |eval| which is the combination of those two
for interpretation.
|compile-file| is currently pretty much tied to a "an output file has
an input file" philosophy. You can define macros before compiling a
file, and add preludes, and even use the c#expand-source and
##expand-source hooks to get to transform the code completely on your
own, but it still assumes that there is an input file, whose path you
need to give to |compile-file| and which has to exist since tha
procedure is going to read from it.
Sometimes you want to build an object file from code coming from
multiple input files (in a more practical way than using macro
definitions in the compiler environment, I think); and sometimes you
want to build multiple object files from one input file, for example
for module parametrization, or for splitting an input file containing
test cases intermixed with normal code into two object files so that
you can (build and) run the tests separately from the normal case.
Creating binary code in the filesystem (as does Gambit) instead of
directly in memory has some advantages: the code blobs can be shared
amongst processes, and they don't have to be rebuilt (the filesystem
is automatically a cache). But we want that as transparent and easy as
possible, too. |compile-file| and |load| is quite good already but
those don't address the following things in their current form:
- recompilation only if needed
- not depend on *input files*
- recompilation of single procedures by entering them in the repl
I want a compile-expression procedure which takes an S-expression as
input (maybe later on, we'll also want it a level deeper, after
lexical parsing). Gambit does have it's internal form of
location-annotated S-expressions already which is just fine; they can
be read from files using ##read-expr-from-port, for example.
I also want that this procedure *only* depends on the input it is
being fed; i.e. it's a pure function in the sense that the resulting
object file only depends on it's inputs. It being pure, you can build
a digest of it's inputs, and count on every output it generated being
the same if the digest is the same. Thus you can use that digest
string as object filename, and if that file already exists, it need
not be generated (compiled), only (re)loaded.
I've started to look into implementing this, but it wasn't easy enough
to do in the couple of partial days I've had to invest so far; I think
especially the part of reloading object files will need some core
work: an object file that has been loaded already doesn't need to be
reloaded, just re-"linked", or whatever it's being called what Gambit
does after loading an object file to make it's toplevel variables
visible in the global symbol table. Reloading an object file which has
already been loaded once means it should just relink those bindings.
Once those are there, you could just redefine |compile-file| and
|load| in terms of those. But use digest strings as object file names
instead, maybe, or whatever you'd like; or write
|compile-file-and-load-if-changed|, or whatever you'd like to use in
your module implementation, anyway. File dependency algorithms
etc. can then even more easily be built on top.
And then the next level can be tackled: lexical parsing and macro
implementation; of course Gambit has those already built in for the
lowlevel kind of macros, and maybe we can use part of that code for
the purpose; parsed trees could then, so I hope, be handled by
hygienic code merging algorithms, and there we might get chances to
make different approaches work together (i.e. code of one module
system can use macros of another one); we'll see.
What I'm also thinking about is that for working with code, we may
want a simpler version of Scheme, one which doesn't know about the
various syntactical sugar forms (like, there is no let or let*, just
define[2] and lambda; letrec may be unavoidable). This should make
writing macro algorithms easier (or manually written lowlevel macros),
right?, or at least other code analyzers (type inferencers,
documentation extraction,..). Kind of an even lower language in a
syntactic tower. I don't know what other Scheme systems do in this
respect but I've read a little bit about GHC's 'core' language, I'm
imagining something in this direction. (And at the same time that it
is simpler than R5RS Scheme it would also have more core 'ops', namely
unsafe ones -- meant to be used safely by using type inferencers/
systems. But from the point of the language, those would still just be
procedures.)
([2] well actually I'm not sure yet whether define could/should be
reduced to lambda and letrec's too, and some sort of more flexible
handling than binding them to "toplevel" variables.)
That's it for now. I'll tell you once I've got some code written; in
the mean time, I'm glad hearing your comments.
Christian.
