Bradley Lucier, or anyone else who knows Meroon,
I figured I would break off from the previous discussion of Meroon to ask this question. It's quite a change of subject.
I'm interested in fixing up Meroon's macros, possibly converting them into a hygienic solution. I'm actually hoping to convert Meroon into a Black Hole module. Black Hole provides DEFINE-MACRO with syntactic closures which provides hygiene. However, that is causing a few problems, mostly to deal with ambiguous situations which would be solved with ER macros. So there is talk of changing Black Hole's macros to maybe use ER macros.
For now, DEFINE-MACRO in Black Hole works pretty well, and I think I can get Meroon working with it. In the future, as Black Hole matures, I can convert its macros into the appropriate hygienic solution which Black Hole provides, preferably syntax-rules since that would be portable.
So that's my task. Can you help me by pointing me in the right direction? I grep'ed Meroon's source for all its macro defines and it looks like it's all in "meroon.gsc" and "macros.scm". However, it looks like Meroon calls certain functions at expansion time, so I need to make sure things are defined in the right phase (expansion, runtime, or both?).
Why does your build script load "pre_meroon.scm" first before it can compile Merooon? Is Meroon written in Meroon? That makes it difficult, as I would have to fix up pre_meroon into a BH module as well, because I can never use BH's macro system and Gambit's at the same time (BH will throw "Invalid syntax" when trying to parse an expression that is a macro but it doesn't know it is).
I've already tried getting it to work, and I had trouble grokking which functions needed to be available in which phase.
Thanks, James
Afficher les réponses par date
I just got really close to getting Meroon working under Black Hole, but it's throwing some runtime errors so obviously it didn't work like I thought.
Basically, I had to insert "syntax-begin" in all sorts of places, extract out macros and defines and make sure the appropriate group of code is evaluates in each expansion phase.
Yuck. Meroon seems to depend on the broken behavior of a macro system without a syntactic tower. Basically when you evaluate one form it becomes available in all expansion phases. This is why it parses through "pre_meroon.scm" and evaluates each expression, so that it loads in all the forms for all phases. Then, it actually runs "_meroon.scm" and macros can call macros and all can reference the same bindings.
I'm probably not using the right terms, and maybe BH is doing something weird too since it doesn't really have a syntactic tower yet.
On Wed, 2010-03-03 at 11:40 -0500, James Long wrote:
So that's my task. Can you help me by pointing me in the right direction? I grep'ed Meroon's source for all its macro defines and it looks like it's all in "meroon.gsc" and "macros.scm". However, it looks like Meroon calls certain functions at expansion time, so I need to make sure things are defined in the right phase (expansion, runtime, or both?).
Why does your build script load "pre_meroon.scm" first before it can compile Merooon? Is Meroon written in Meroon? That makes it difficult, as I would have to fix up pre_meroon into a BH module as well, because I can never use BH's macro system and Gambit's at the same time (BH will throw "Invalid syntax" when trying to parse an expression that is a macro but it doesn't know it is).
I've already tried getting it to work, and I had trouble grokking which functions needed to be available in which phase.
James:
I'm really impressed with the progress you've made with Meroon in such a short time.
Let me explain my understanding of how Meroon is built. Even though I have not been able to complete a dependency graph, I believe I understand how it works. To my mind, it is rational, it may not be pretty ;-).
Yes, Meroon is built in Meroon. Preliminary classes are built by hand using low-level constructors, and then the high levels of Meroon are built on top of that.
If you want to run with an interpreted Meroon, to debug the Meroon sources, say, you can just run the make_meroon script to build _meroon.scm and say:
heine:~/Desktop/MeroonV3-2008Mar01> gsi Gambit v4.6.0
(load "old-load")
"/home/lucier/Desktop/MeroonV3-2008Mar01/old-load.scm"
(old-load "_meroon.scm")
[ Meroon V3 Paques2001+1 $Revision: 1.2 $ ]
So now Meroon is available to you, and everything is being run interpreted. pre_meroon.scm isn't needed.
This will not work with Gambit's load procedure, you need to use the old-load procedure, as defined in old-load.scm, which simulates a repl, it reads each form in _meroon.scm in order and executes it, whether that form defines a macro, a function, etc. When using old-load, gambit first expands each form using any macros that have been defined previously, which can use any functions that have been defined previously. So this is a layered build system, with very thin layers ;-).
Gambit's load procedure won't work to load _meroon.scm all at once, because when Gambit loads a file it first reads all the forms in the file, evaluates the macro definitions in the file (which may use any functions defined in previous files, not functions from the current file), expands the rest of the source using all now-existing macros and functions, and then loads all the non-macro forms in the file.
So one could determine which forms in _meroon.scm depend on which macros and on which functions, and group them in files like:
F0: functions that don't depend on any macros M0: Macros that use only the functions in F0 (and standard functions) to expand forms F1: functions that can be expanded by the macros in M0 (and standard macros) M1: Macros that use the functions in F0 and F1 and the macros in M0 to expand forms etc.
and then you could load F0, M0, F1, M1, F2, M2, ... in order.
So then the question is: What is the role of pre_meroon?
I asked Christian to make a number of changes to Meroon, which he did in a way that made minimal changes to the Meroon sources---he added a file postset.scm. This file defines a new base class, Inlinable-Class, that generates accessors, predicates, and makers in a form that the Gambit compiler can inline. It also changes the name of setters from set-classname-fieldname! (in the style of set-car!) to classname-fieldname-set! (in the style of vector-set!). The default base class is then set to Inlinable-Class.
Very cool. Demonstrates the meta-object protocol of Meroon. But all the previous code defining Meroon uses setters with names in the style set-classname-fieldname! (and the previously default Class), so once you load postset.scm, the Meroon you've just built won't recognize the code that was used to build it.
So _meroon.scm is pre_meroon.scm with postset.scm appended to it. pre_meroon.scm (without postset.scm) is loaded form by form; then _meroon.scm is compiled in one go (with all the macros and functions defined by pre_meroon.scm used to expand forms in _meroon.scm before compilation). Then you can load directly into gsi or gsc the compiled file _meroon.o1 (which, at the end of it, changes the default class to Inlinable-class).
If you try to compile _meroon.scm after loading _meroon.scm, you get the following problem:
heine:~/Desktop/MeroonV3-2008Mar01> gsc Gambit v4.6.0
(load "old-load.scm")
"/home/lucier/Desktop/MeroonV3-2008Mar01/old-load.scm"
(old-load "_meroon.scm")
[ Meroon V3 Paques2001+1 $Revision: 1.2 $ ]
(compile-file "_meroon.scm")
"/home/lucier/Desktop/MeroonV3-2008Mar01/_meroon.o2"
*** EOF again to exit heine:~/Desktop/MeroonV3-2008Mar01> gsc Gambit v4.6.0
(load "_meroon")
[ Meroon V3 Paques2001+1 $Revision: 1.2 $ ]
******************* Meroon Domain ********************* Occurred in: allocator Reason: Inappropriate class Hint[0]: #<Class: Class> Hint[1]: #!unbound2 *** ERROR IN Mono-Field-create-careful-reader -- Meroon "Error" 1>
So, Meroon is built in a layered system, and this system could be made more explicit. There is no confusion (in my mind) about phases---whenever a form is initially loaded, it is expanded using any previously defined macros and functions, then the expanded form is loaded into the interpreter. We need this little trick to compile _meroon.scm, which includes postset.scm.
Sorry, I don't have more time to make this e-mail shorter.
Brad
On Wed, Mar 3, 2010 at 2:24 PM, Bradley Lucier lucier@math.purdue.edu wrote:
James:
I'm really impressed with the progress you've made with Meroon in such a short time.
Thanks! I've got some work lined up that I really, really want to use Black Hole + Meroon for. So I've got to get it working.
If you want to run with an interpreted Meroon, to debug the Meroon sources, say, you can just run the make_meroon script to build _meroon.scm and say:
Good to know!
This will not work with Gambit's load procedure, you need to use the old-load procedure, as defined in old-load.scm, which simulates a repl, it reads each form in _meroon.scm in order and executes it, whether that form defines a macro, a function, etc. When using old-load, gambit first expands each form using any macros that have been defined previously, which can use any functions that have been defined previously. So this is a layered build system, with very thin layers ;-).
The problem is that this is assuming something very specific about how Gambit expands things. This makes is very difficult to port it to any module system which Gambit might adopt. Because it isn't properly laid out with clear semantics between phases, there is much work to do to port it to a good module system which separates phases into completely disparate environments.
So one could determine which forms in _meroon.scm depend on which macros and on which functions, and group them in files like:
F0: functions that don't depend on any macros M0: Macros that use only the functions in F0 (and standard functions) to expand forms F1: functions that can be expanded by the macros in M0 (and standard macros) M1: Macros that use the functions in F0 and F1 and the macros in M0 to expand forms etc.
and then you could load F0, M0, F1, M1, F2, M2, ... in order.
Yes, that would be nice. I'd rather not learn *that* much about Meroon, so if anyone wants to do that, I'll buy them a beer. However, in the end, maybe I'll do that over time. :)
So then the question is: What is the role of pre_meroon?
pre_meroon is more than that. If that's all it was (a plain Meroon system), I should be able to compile it with `gsc pre_meroon.scm`. But I can't, because it depends on the strange behavior of loading and evaluating each form at a time. So, by loading it that way, and then using it to compile _meroon.scm, it is also a bootstrapping mechanism, without which you wouldn't be able to compile Meroon at all, with or without the postset changes.
See:
% gsc pre_meroon *** ERROR IN feature-present? -- Unbound variable: *meroon-features*
Looking at the code, it's obvious, because it looks something like this:
(define *meroon-features* '(gambit ...))
(define-macro (if-meroon-feature thing) (if (member thing *meroon-features*) ...))
The macro obviously uses a variable that is clearly defined for the runtime environment. So, while compiling _meroon.scm, it is actually using these global variables defined in per_meroon.scm, which is just odd.
It gets worse. There is some core global state in Meroon, such as:
(define *class-number* 0)
This variable is referenced by code both in the runtime and macro phases. Now, because of the way Meroon is loaded/compiled, this happens to work and point to the same data. But I'm not sure I see much hope anymore of getting Meroon to work with a proper module system. The macro environment is (and should be) completely separate from the global environment in a proper module system.
Basically, there's a macro (CHECK-AND-PROCESS-CLASS-DEFINITION) which modifies *class-number*, but *class-number* is also modified in several functions.
I'd like to get this working, but am I wrong in saying that it's going to be a lot of work? Are my conclusions valid? Thanks for being responsive!
- James
On Wed, Mar 3, 2010 at 4:15 PM, James Long longster@gmail.com wrote:
Basically, there's a macro (CHECK-AND-PROCESS-CLASS-DEFINITION) which modifies *class-number*, but *class-number* is also modified in several functions.
Actually, looks like I was wrong. From some testing I was doing, that looked to be the case, and I misread the code. Luckily, this isn't the case, so maybe it is just a matter of separating out the code into more explicit layers that we can load as modules.
Well, I wanted to follow up to my previous postings. I am now going through Meroon's sources in a proper manner, instead of trying to get it to work while being ignorant of how it actually works.
I was pretty ignorant. It looks like Meroon purposefully keeps track of state during macro expansion so that it can do a lot of weight- lifting and optimizations during macro expansion.
I'm pretty impressed. I'm still trying to figure out how to port it to a module system with a syntactic tower (or at least 2 different environments for expansion and runtime). This will take time. But if I learn how Meroon works, I should be able to do it properly.
Sent from my iPhone
On Mar 3, 2010, at 4:33 PM, James Long longster@gmail.com wrote:
On Wed, Mar 3, 2010 at 4:15 PM, James Long longster@gmail.com wrote:
Basically, there's a macro (CHECK-AND-PROCESS-CLASS-DEFINITION) which modifies *class-number*, but *class-number* is also modified in several functions.
Actually, looks like I was wrong. From some testing I was doing, that looked to be the case, and I misread the code. Luckily, this isn't the case, so maybe it is just a matter of separating out the code into more explicit layers that we can load as modules.
Hi,
2010/3/4 James Long longster@gmail.com:
Bradley Lucier, or anyone else who knows Meroon,
It's been a while since I last read it, plus I don't have it here, but I'm pretty sure that Meroon (or a look-alike) is described in the last chapters of Lisp in Small Pieces, by Christian Queinnec. Perhaps you could find some useful facts or piece of information about Meroon in this bible. By the way, did Christian Queinnec completely stop maintaining Meroon, or did you not try to ask him questions about it?
Cheers,
P!
Hallo,
On Wed, Mar 3, 2010 at 10:01 PM, Adrien Piérard pierarda@iro.umontreal.ca wrote:
Hi,
2010/3/4 James Long longster@gmail.com:
Bradley Lucier, or anyone else who knows Meroon,
It's been a while since I last read it, plus I don't have it here, but I'm pretty sure that Meroon (or a look-alike) is described in the last chapters of Lisp in Small Pieces, by Christian Queinnec. Perhaps you could find some useful facts or piece of information about Meroon in this bible.
L.i.S.P. comes with a simpler version of Meroon called Meroonet. It lacks much of the MOP.
Cheers,
-
Adrien Piérard -
Alex Queiroz -
Bradley Lucier -
James Long