On 2013-02-20, at 10:58 AM, Jason Felice <jason.m.felice@gmail.com> wrote:

>
> I've had a couple ideas for modifications to the Scheme reader that I imagine would be front-ends to the compiler.  Examples:
>
> 1. A front end which reads Scheme source but specially handles square braces, transforming them into Objective-C method calls via my library, so that I can put the following in a program:
>
>    (let ((s [[NSString alloc] initWithCapacity:42]))
>       ...)
>
> And have it dispatch non-dynamically (my current project is completely dynamic).

Extending the reader to handle square brackets is easy because the reader already has hooks to customize the reading of [...], {...}, <...>, and (...).  Here's an example:

 Gambit v4.6.7

 > (include "~~lib/_gambit#.scm")
 > '[NSString alloc]
 (NSString alloc)
 > (macro-readtable-bracket-keyword-set! ##main-readtable 'ObjC-send)
 > '[NSString alloc]
 [NSString alloc]
 > (length '[NSString alloc])
 3
 > (car '[NSString alloc])
 ObjC-send
 > (cadr '[NSString alloc])
 NSString
 > (caddr '[NSString alloc])
 alloc
 > (list->vector '[NSString alloc])
 #(ObjC-send NSString alloc)
 > (define-macro (ObjC-send receiver . args) `(pp 'objective-c-send...))
 > [NSString alloc]
 objective-c-send...

Another example, which treats [...] as a vector constructor is:

 > (include "~~lib/_gambit#.scm")
 > (macro-readtable-bracket-keyword-set! ##main-readtable 'vector)
 > [1 2 3 4]
 #(1 2 3 4)

> 2. A testing library where "check" forms appear under each function in the same file as documentation.  The check forms normally expand to nothing, but a command can be run from the command line scans the current directory and sub-directories for .scm files which (check ...) forms in them and runs them.

This should be easy to do with macros.

> 3. A complete environment with support for a lot of SRFI's and utility functions built in - a new REPL/compiler which is specialized for some domain.
>
> Black hole appears to do something like this.
>
> My question is, how do I hook into Gambit to compile or interpret after I've transformed the code, preserving line numbers?

When you transform the code you must preserve the location information that was generated by the reader.

The interpreter and compiler invoke the reader in a special mode where each datum (and subdatum) that is read is wrapped in a structure which gives the location information.  These structures are called "source" objects, and the ##make-source function is the constructor.

Here is an example.  Say you have a file "foo.scm" with the content:

 (define n 1)
 (pp n)

The interpreter and compiler will read that source code using the ##read-all-as-a-begin-expr-from-path function, which calls Gambit's extended read function to read each expression and construct a begin expression:

 > (vector-ref (##read-all-as-a-begin-expr-from-path
                "foo.scm"
                (##current-readtable)
                (lambda (re x) x)
                (lambda (re x) x))
               1)
 (##begin (define n 1) (pp n))

The last two parameters of ##read-all-as-a-begin-expr-from-path are the wrap and unwrap functions (whose first argument is the read-environment, which contains the reader's state including the port being read from and the current line/column location).  By supplying the predefined ##wrap-datum and ##unwrap-datum functions, each datum (and subdatum) will be wrapped in a source object:

 > (vector-ref (##read-all-as-a-begin-expr-from-path
                "foo.scm"
                (##current-readtable)
                ##wrap-datum
                ##unwrap-datum)
               1)
 #(#(source1)
   (#(#(source1) ##begin "/Users/feeley/gambit/work/foo.scm" 0)
    #(#(source1)
      (#(#(source1) define "/Users/feeley/gambit/work/foo.scm" 65536)
       #(#(source1) n "/Users/feeley/gambit/work/foo.scm" 524288)
       #(#(source1) 1 "/Users/feeley/gambit/work/foo.scm" 655360))
      "/Users/feeley/gambit/work/foo.scm"
      0)
    #(#(source1)
      (#(#(source1) pp "/Users/feeley/gambit/work/foo.scm" 65537)
       #(#(source1) n "/Users/feeley/gambit/work/foo.scm" 262145))
      "/Users/feeley/gambit/work/foo.scm"
      1))
   "/Users/feeley/gambit/work/foo.scm"
   0)

The source objects are vectors with the special tag #(source1) as their first element, i.e. #(#(source1) <datum> <filename> <column*65536+line>) .  Note that when <datum> is a list or vector, each of the elements is wrapped in a source object, and so on recursively.  The datum can be extracted with ##source-code, and the location information with ##source-locat.

The interpreter and compiler can maintain this source location information when macros are defined with the ##define-syntax special form.  The macro transformer function receives a source object with location information, and can return a source object with location information (the transformer can also return a non source object, which will get automatically wrapped with a source location object with the location of the macro call).  Here's an example:

 > (define (where source)
    (##display-locat (##source-locat source) #f (repl-output-port))
    (newline))
 > (##define-syntax foo (lambda (source) (pp source) (where source) #f))
 > (foo 1 2)
 #(#(source1)
   (#(#(source1) foo (console) 65540)
    #(#(source1) 1 (console) 327684)
    #(#(source1) 2 (console) 458756))
   (console)
   4)
 (console)@5:1
 #f

Marc