Hi Alvaro,

(Two Q:s for you at the bottom.)

2013/1/25 Álvaro Castro-Castilla alvaro.castro.castilla@gmail.com

Hi!

I write about the 2 latest emails here:

• Separate module-systems compilation:

That's it. I think that is the best solution. I already did a proof of concept in the past for scheme spheres and it works well: some modules are expanded with one expander and some with other. The result is taken by Gambit seamlessly. In my experience, people don't mix define-macros and syntax-rules. For those cases where you *NEED* define-macros (like processing names of identifiers) a second pass could be offered, but is another story. One simple expander per module is a modular and simple solution I'm totally in for.

Yeah.

By this do you mean that you think the idea I proposed in the email ~17hrs ago is a good direction for the development?

• Lightweight-portable implementation of BH:

You can see some of the techniques I talk about here: https://www.youtube.com/watch?v=Q7c0rU9Lv28

The point is that if you have an implementation that is lightweight and pluggable enough, you can use it for remote REPLs of any kind (mobile debugging, web servers, hot code swapping in webservers -which can be done with termite-), like the example of the video.

Why I so much like Alexpander is because it does exactly that: it is less than 1KLOC and works with any scheme system. The only issue with being portable is that you don't have that nice error reporting, apparently (and that you don't use the non-portable facilities that could make it faster and more compact). But BH doesn't need to be portable across scheme implementations so this is our chance to make a BH with these features.

This is why I insist so much on the separation of concerns and the lightest possible BH. I've been thinking about possible ways of making BH just a text processor (like Alexpander currently is), and I believe is possible. The only issue is that probably you need to output more than one file. Simply knowing which symbols to prepend with which namespace prefix, you already have a simple module system that can have the same encapsulation properties than BH, if you mix that with macro expansion then you have all you need.

The only thing I'm trying to figure out is how to make this #namespace be prepended to macros names, as they need to be part of the module system as well, but that could be handled by the macro expander (which goes hand in hand with this module system).

Really, what I have now working at Scheme Spheres and works pretty well (waaaaay faster macro expansion than BH) gives me this:

• full expansion of syntax-rules in a first pass
• expansion of define-macro's in a second pass
• module system in the Gambit style
• fully embeddable in any system that I want to not just eval scheme

code, but process its macros

What does it need to achieve all we need from Blackhole and be superior due to the portability and embeddability?

• module system that incorporates the macros
• better error reporting for macros

In summary, a solution like I'm proposing would have the following features:

• embedabbility: could be embedded in a web browser (with a javascript

backend), in a web server for hot code swapping and testing, live coding (see http://toplap.org/ for some examples of this world, and let me tell you that one of the most interesting apps for live coding is currently running on Racket), remote debugging and coding of Android devices. And many others options that this would open the doors to.

• composability: a separation of concerns would make it easy for

people to make small apps that glue together a workflow, very much in the Unix style. Also, this would be more welcoming for people to contribute to new "utilities". For instance, we now should worry just about the essential utilities (macros, modules and maybe packages), but all the niceties that we find in other ecosystems can be built on top of these 3 (and I'm talking about gem/sids, bundler/sbundle, testing, literate programming, automatization). And these utilities could be used by people who don't want to use BH at all!

• control: simplicity and respecting the "Gambit ways" (like just

producing a Scheme output and delegate to Gambit), will allow for *full* control of how you want your code to live.

I mean, with BH we lost control of things like dynamically loading object files by need,

What was this about, can you describe the use case and problem?

or just compiling everything in one C file for MAX performance (avoiding trampolines).

I believe BH has a feature for this, it's something like that it can take all the modules at least in the same dir and make only one module file out of them.

Well, that is my proposal, and I believe it is actually much simpler

than original Blackhole and would solve more problems.

Yes, it looks like it will be something like this.

Best regards,

Álvaro

So basically the Scheme environment-specific code (like production of ##namespace codes, inserting code into the Scheme environment, doing compile-file) should be split apart from as much of the module system 'core' as possible.

Maybe, I'll start drafting some kind of API for all this stuff soon.

* Would be happy if you could give spontaneous feedback on the API draft I sent you yesterday, for interface between package management/module loader and module system.

I'm not clear on what kind of requirements that a live coding environment or alike could have.

Really, this BH now will be made primarily for Gambit, and support for generation toward other systems will be a bit like a bonus. I believe something like a-set-of-modules => a Scheme code image suitable for feeding Scheme2JS with, is completely realistic.

* Alvaro: When do you believe your document on use and advanced use of syntax-rules will be ready?

I'm a but curious on how line number info should be managed through this expansion process. I know Gambit has an internal format for it already, hopefully we can just hook onto that.

All the macro expanders will need to traverse the input Scheme structure atop a DSL that deals with this format.

Really, if the DSL:s involved are just well documented, and all the involved modules can be loaded as ordinary BH modules for anyone who wants to use them, then I believe we've got a high degree of plugability and reusability and dynamicity in all kinds of directions already there.

So, BH was monolithic in these respects, perhaps it could have been designed like this but it was not. Hopefully by just doing this we get all those godo qualities.

Brgds, Mikael