syntax-case with compiled syntax and modules - Gambit-list

11 Sep 2013


      Gambit people,
I had noticed that the shipped syntax-case in Gambit 4.7.0 has issues with supporting compiled syntax and modules. My experience has been that although one can compile such things, loading compiled objects into a fresh repl has resulted in failure:
;; hello.scm
(module hello
        (hello-world)
        (define (hello-world)
          "hello, world"))
;; nth-value.scm
(define-syntax nth-value
  (syntax-rules ()
    ((nth-value n values-producing-form) (call-with-values
                                             (lambda () values-producing-form)
                                           (lambda all-values
                                             (list-ref all-values n))))))
Gambit v4.7.0
...
(compile-file "hello.scm")
"/Volumes/Data/scratch/scheme/control/home/hello.o1"
...
(load "hello")
"/Volumes/Data/scratch/scheme/control/home/hello.o1"
...
(import hello)
hello-world
#<procedure #2 _0>
...
(hello-world)
"hello, world"
...
,q
Process scheme finished
Gambit v4.7.0
...
(load "hello")
"/Volumes/Data/scratch/scheme/control/home/hello.o1"
...
(import hello)
*** ERROR -- unknown module hello
...
(compile-file "nth-value")
"/Volumes/Data/scratch/scheme/control/home/nth-value.o1"
...
(load "nth-value")
"/Volumes/Data/scratch/scheme/control/home/nth-value.o1"
...
(nth-value 1 (values 'a 'b))
b
...
,q
Process scheme finished
Gambit v4.7.0
...
(load "nth-value")
"/Volumes/Data/scratch/scheme/control/home/nth-value.o1"
...
(nth-value 1 (values 'a 'b))
*** ERROR IN (console)@2.2 -- Unbound variable: nth-value
1>
A little debugging has shown root cause to be a lack of generated environment information in the sc-expand compiler output. All  module metadata and syntax-definitions is never emitted in the compiled output. This is exemplified by adding some pretty print to both ##expand-source and c##expand-source definitions:
(set! c#expand-source ;; setup compiler's expander
(lambda (src)
(let ((r (sc-expand src)))
  (pp 'compiler)
  (pp (##desourcify r))
  r)))
(set! ##expand-source ;; setup interpreter's expander
(lambda (src)
(let ((r (sc-expand src)))
  (pp 'interpreter)
  (pp (##desourcify r))
  r)))
With side-effects added, one can see the differences in output during both interpretation, where these features work, and compilation, where they are busted.
...
(compile-file "hello.scm")
interpreter
(compile-file "hello.scm")
interpreter
($sc-put-cte
 '#(syntax-object
    ##compilation-options
    ((top)
     #(ribcage #(##compilation-options) #((top)) #(##compilation-options))))
 '(global . ##compilation-options)
 '*top*)
interpreter
(define ##compilation-options '((target c)))
interpreter
($sc-put-cte
 '#(syntax-object hello ((top) #(ribcage #(hello) #((top)) #(hello))))
 '($module . #(interface
               (top)
               #(#(syntax-object
                   hello-world
                   ((top) #(ribcage #(hello-world) #((top)) #(_2)))))
               _3))
 '*top*)
compiler
(begin
  (define _2 (void))
  (letrec ((%%_21 (lambda () "hello, world"))) (begin (set! _2 %%_21) (void))))
"/Volumes/Data/scratch/scheme/control/home/hello.o2"
...
When one examines the output, it can be verified that the $sc-put-cte construct, which defines the module, is never emitted in the compiled object. I found a similar result for define-syntax constructs.
It turns out that the modifications to make compiled syntax and modules work is trivial, and so I've included them in the attached patch, which includes the following alterations to psyntax73.ss:
- Modification to generate-id to append the original symbol name in generated symbols. I'm hoping this might help with the debugging of macros, where one can see _0.foo in lieu of the current _0, when foo is mapped to _0.
- The addition of a sc-compile-expand peer to sc-expand that binds '(L C) and '(L) values for ctem and rtem. With these alternative bindings, the syntax-case expander emits $sc-put-cte forms in compiled objects. Users should note that this inserts a dependency on syntax-case.scm in compiled binaries, which was not formerly present.
- The alteration of build-cte-install to sourcify syntax-object definitions. I have found that the c##expand-source implementation is particularly sensitive to non-sourcified content, and tends to produce invalid compilation output or segmentation fault when it receives such. This modification was needed to present the Gambit compiler with the sourcified content needed for bitchin' smooth emissions.
To use the patch, apply the changes herein to the gambit/lib source directory, run syntax-case-build, and make install to copy the regenerated syntax-case.scm into your gambit installation. The syntax case expander is then enabled by running gambit in 'standard scheme mode', as documented in the manual.
Kind regards,
Matt Hastie.