I agree with Meng. I see syntax-rules as a DSL for hygienic macros. It is completely "schemey" in the same way libraries like Kanren for logic programming, or FrTime for reactive programming are. The only difference is that when using hygienic macros, your code becomes data as well, to be processed before it actually turns into code. Actually, when you take into account a couple of pitfalls regarding lexical scoping and shadowing and use a variety of techniques, including continuation-passing-style, writing syntax-rules macros are extremely powerful and similar to regular recursive scheme. Unhigienic macros are well-known timebombs that are waiting to explode as soon as client code does something the library didn't think of. They are useful for self-sufficient systems, as Mikael said.

Meng, I wasn't aware of this "riaxpander", it seems chicken also has it. Is your riaxpander implementation open source?

I would add these comments to the original post:

- CAR and CDR are shorter than FIRST and REST (why not HEAD-TAIL?), so besides the historical meaning, I prefer them for this reason. However, you can always define your own first and rest (the former is actually defined in SRFI-1). About making them generic, next point: - I think the procedure specialization for types (char=, *-lenght, etc...) is good as it favors performance. If you want the generic ones, it is straightforward to define. For instance, that's what the author of SRFI-47 does: replace array=? with an array-augmented version of R5RS equal?. While you can do your own specialization, you couldn't do it the other way around: given a generic procedure in R5RS, specialize it for your types. - Mikael's point about symmetry is absolutely beautiful. And indeed I defend the usefulness of values, which are of special interest in functional programming where you avoid side-effects. Also, the points about C/C++ (and assembly I may add) are completely true, but that's one of the reasons I find Gambit a particularly powerful system. - Generics are defined in several libraries, there are many implementations. - Promise and force are re-defined in R5RS terms in SRFI-45. I agree with this point, but I don't know the deeper reasons why they are included, because even opening the possibility to implementations-defined optimized representation of this primitives could have been done with an SRFI.

Thanks to everyone for your comments.

Best regards

On Sat, Jan 5, 2013 at 1:16 PM, Meng Zhang wsxiaoys.lh@gmail.com wrote:

Great post Mikael! So happy to read your thoughts.

On Sat, Jan 5, 2013 at 6:37 PM, Mikael mikael.rcv@gmail.com wrote:

...

The syntactic-closures based macro systems (syntax-rules/syntax case including define-syntax, syntax-rules, let-syntax, letrec-syntax) bring incredible complexity and with that low debuggability through a very complex identifier concept based a kind of duality of the identifier symbol in itself and the syntactic environment in which it is used in a particular instance, and brings a complex API for handling macros with this, that by nature is not Schemy and not suited for debugging.

Per came with the suggestions above based on having spent approx 6 months fulltime on developing the Black Hole module system, which does hybrid define-macro and syntactic closures expansion.

There might be some caveat I didn't get, but, I'd guess you could actually make a macro expander that supports both an alias macro system as per above and a syntactic-closures for compatibility with code that uses it, possibly by splitting expansion into two expansion phases, thus isolating all the identifier-related complexity.

While talking about the syntactic-closures, If we ignore syntax-case, I'll disagree that it brings "incredible" complexity. I've been uncertain on syntactic-closures for years, while this Holiday I finally got spare time reading its implementation in riaxpander/chibi-scheme, I found the concept of it is quite straight forward. Though I deadly missed an easy-to-understand, concrete implementation with document for it during the learning process.

I've cloned such a system in gambit, and keep digging on how to integrate hygiene with module system and gambit's compiling process.

Thanks Meng

---

Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list

Hi Jason,

(Just so I got you right, by generality of built-in procedures you do actually mean generics, as in, different actual routines are invoked based on type?)

I believe something like a generics - multiple dispatch system would be really handy.

Through algorithms for analysis through abstract interpretation like 2-CFA, you can probably pin down quite a lot of data types - maybe even 95% for typical-ish code - and thus what exact procedures to use at expansion time and thus also get a really good performance.

Prof. Matt Might at Utah works with abstract interpretation, perhaps he published an implementation of a Scheme-based abstract interpreter of Scheme.

I suppose implementing this atop standard Scheme makes good sense.

2013/1/7 Jason Felice jason.m.felice@gmail.com

Thanks for the feedback. I appreciate it. I think I'd like to narrow my focus to the primary thing, which was the lack of generality of the built-in procedures, and generics.

Making these efficient to compile (especially with a module system) seems hard, but rewarding. I'm thinking about how to do this more. Personally, I don't think efficiency in terms of constant factors should often win versus code which could be more general.

Clearly this is a value choice; however, I wonder how well a compiler can eliminate type dispatching without adding type annotation to the language.

Without having studied it too deep, I believe abstract interpretation can help you a lot with this. If you rely on that objects have a type slot to them, like Gambit's define-type:s have (it's ##vector-ref :able on slot 0 or 1), then you can always dig this out during runtime and handle it the correctly then.

So, I'm thinking about these things and will let them stew for... another 10 years? I don't know.

Thanks, -Jason

Please feel free to share reflections you're coming up with along the way.

Brgds, Mikael

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1

On 08-01-13 19:18, Hendrik Boom wrote:

On Mon, Jan 07, 2013 at 10:37:08AM -0500, Jason Felice wrote: ... ...

...

I don't think efficiency in terms of constant factors should often win versus code which could be more general. Clearly this is a value choice; however, I wonder how well a compiler can eliminate type dispatching without adding type annotation to the language.

For me, the value of type anotations is the possibility of static type checking, which catches bugs fast. That the compiler can then use the information to generate better code is a pleasant freebie.

It would be interesting to see if the stragegy in, say, typed Racket, could be usefully adapted to Gambit.

Bigloo is another older example of a Scheme with optional type annotations.

Marijn

2013/1/5 Meng Zhang wsxiaoys.lh@gmail.com

While talking about the syntactic-closures, If we ignore syntax-case, I'll disagree that it brings "incredible" complexity. I've been uncertain on syntactic-closures for years [..] I found the concept of it is quite straight forward.

2013/1/6 Álvaro Castro-Castilla alvaro.castro.castilla@gmail.com

I agree with Meng. I see syntax-rules as a DSL for hygienic macros. It is completely "schemey" in the same way libraries like Kanren for logic programming, or FrTime for reactive programming are. The only difference is that when using hygienic macros, your code becomes data as well, to be processed before it actually turns into code.

Meng and Alvaro, ah, note taken, to have a clearer take on them personally, I'd need some serious experience of using them, and until now I did not really dig into them that much personally.

It's really unfortunate, in a sense, that in a hybrid S.C. and define-macro environment, that any use in define-macro of symbol? or any kind of symbol inspection or comparison, easly becomes *Completely Messed*.

Here's a mild example:

(define-macro (debug-print . a) (for-each (lambda (e) (cond ((symbol? e) (print "'" (symbol->string e) " ")) ((string? e) (print e " ")) (else (write e) (print " ")))) a))

(debug-print 5 + 7 "\n")

What is printed in the place of "+" depends on the particular grace of the SC macros loaded and the expander in this moment and instance.

It could |write| any expander-internal structure really, with any amount of object dependencies.

Now this was a really basic example, if some more extensive one comes to your mind feel free to share.

Meng:

Though I deadly missed an easy-to-understand, concrete implementation with document for it during the learning process.

Yes! Like, a complete reference of Scheme macro systems: How to use them, strengths and limitations, how to implement them on their own and in hybrid forms, and what the challenges are, including illustrative examples.

This would do well as a downloadable PDF book.

Alvaro:

Actually, when you take into account a couple of pitfalls regarding lexical scoping and shadowing and use a variety of techniques, including continuation-passing-style, writing syntax-rules macros are extremely powerful and similar to regular recursive scheme.

Can you give any code examples of use of this variety of techniques for writing extremely powerful syntax-rules macros?

2013/1/8 Hendrik Boom hendrik@topoi.pooq.com

On Mon, Jan 07, 2013 at 10:37:08AM -0500, Jason Felice wrote: ... ...

...

I don't think efficiency in terms of constant factors should often win versus code which could be more general. Clearly this is a value

choice;

...

however, I wonder how well a compiler can eliminate type dispatching without adding type annotation to the language.

For me, the value of type anotations is the possibility of static type checking, which catches bugs fast. That the compiler can then use the information to generate better code is a pleasant freebie.

It would be interesting to see if the stragegy in, say, typed Racket, could be usefully adapted to Gambit.

Yes, type annotations that catch bugs already at expand time would be of value.

The abstract interpreter might not catch all at compile time of course, but, really enough. I'd be curious to know what kind of performance such a solution would have .. I wonder what Might published recently.

Hi Steve,

That's interesting. I would guess this has nothing to do with the macro facility or this code per se, but that this behavior was from some completely other source, like, the general functioning of the console driver or something. Thoughts?

M

2013/1/11 Steve Graham jsgrahamus@yahoo.com

I tried typing this code several times into Gambit on my Samsung phone. The first several times it simply defaulted to the phones default screen and each time when I invoked Gambit it would act as it should. This last time it we to the normal Gambit screen, but all that's showing is the header line with a white screen. I believe the last time it opted to send a report, which I did. Advanced Task Killer does not solve the problem.

Restarting the phone does solve this problem.

..

(define-macro (debug-print . a) (for-each (lambda (e) (cond ((symbol? e) (print "'" (symbol->string e) " ")) ((string? e) (print e " ")) (else (write e) (print " ")))) a))

(debug-print 5 + 7 "\n")

I've been planning to write an extense document about syntax-rules, since there is a variety of techniques that aren't documented anywhere except in the mailing lists or usenet groups. Actually, you can even write unhygienic macros using syntax-rules, but that's a whole different story (and the paper describing the technique is 15 pages showing you how to break the system).

I'll try to write that document as soon as possible!

On Fri, Jan 11, 2013 at 5:25 PM, Mikael mikael.rcv@gmail.com wrote:

2013/1/5 Meng Zhang wsxiaoys.lh@gmail.com

...

While talking about the syntactic-closures, If we ignore syntax-case, I'll disagree that it brings "incredible" complexity. I've been uncertain on syntactic-closures for years [..] I found the concept of it is quite straight forward.

2013/1/6 Álvaro Castro-Castilla alvaro.castro.castilla@gmail.com

...

I agree with Meng. I see syntax-rules as a DSL for hygienic macros. It is completely "schemey" in the same way libraries like Kanren for logic programming, or FrTime for reactive programming are. The only difference is that when using hygienic macros, your code becomes data as well, to be processed before it actually turns into code.

Meng and Alvaro, ah, note taken, to have a clearer take on them personally, I'd need some serious experience of using them, and until now I did not really dig into them that much personally.

It's really unfortunate, in a sense, that in a hybrid S.C. and define-macro environment, that any use in define-macro of symbol? or any kind of symbol inspection or comparison, easly becomes *Completely Messed*.

Here's a mild example:

(define-macro (debug-print . a) (for-each (lambda (e) (cond ((symbol? e) (print "'" (symbol->string e) " ")) ((string? e) (print e " ")) (else (write e) (print " ")))) a))

(debug-print 5 + 7 "\n")

What is printed in the place of "+" depends on the particular grace of the SC macros loaded and the expander in this moment and instance.

It could |write| any expander-internal structure really, with any amount of object dependencies.

Now this was a really basic example, if some more extensive one comes to your mind feel free to share.

Meng:

...

Though I deadly missed an easy-to-understand, concrete implementation with document for it during the learning process.

Yes! Like, a complete reference of Scheme macro systems: How to use them, strengths and limitations, how to implement them on their own and in hybrid forms, and what the challenges are, including illustrative examples.

This would do well as a downloadable PDF book.

Alvaro:

...

Actually, when you take into account a couple of pitfalls regarding lexical scoping and shadowing and use a variety of techniques, including continuation-passing-style, writing syntax-rules macros are extremely powerful and similar to regular recursive scheme.

Can you give any code examples of use of this variety of techniques for writing extremely powerful syntax-rules macros?

2013/1/8 Hendrik Boom hendrik@topoi.pooq.com

...

On Mon, Jan 07, 2013 at 10:37:08AM -0500, Jason Felice wrote: ... ...

...

I don't think efficiency in terms of constant factors should often win versus code which could be more general. Clearly this is a value

choice;

...

however, I wonder how well a compiler can eliminate type dispatching without adding type annotation to the language.

For me, the value of type anotations is the possibility of static type checking, which catches bugs fast. That the compiler can then use the information to generate better code is a pleasant freebie.

It would be interesting to see if the stragegy in, say, typed Racket, could be usefully adapted to Gambit.

Yes, type annotations that catch bugs already at expand time would be of value.

The abstract interpreter might not catch all at compile time of course, but, really enough. I'd be curious to know what kind of performance such a solution would have .. I wonder what Might published recently.