When translating Clojure code to Scheme, I'm finding it quite easy to preserve source code information for the Gambit interpreter, so that when I get exceptions I am notified of the offending Clojure source lines. Is there an obvious way I could do this for compiled code as well?
Currently, I can emit location-tagged code in the '#(##source2-marker code file locat) form which can be evalled by gsi directly. I expect I may need to write a modified version of gsc which is able to read these forms without adding its own line numbering annotations. Another option is to write a version of gsc with the whole Clojure->Scheme translation process built in.
Basically, I'm expecting to write a variant of gsc with a modified "compile-program" function that does not call the "read-source" function but my own "read-annotated-source" function. I just want to confirm I am not missing a compiler flag or some other method of hooking into gsc's reader.
Afficher les réponses par date
On May 31, 2013, at 9:28 AM, Nathan Sorenson takeoutweight@hotmail.com wrote:
When translating Clojure code to Scheme, I'm finding it quite easy to preserve source code information for the Gambit interpreter, so that when I get exceptions I am notified of the offending Clojure source lines. Is there an obvious way I could do this for compiled code as well?
There isn't a form for this, but that would be useful and I have thought about adding it. Something like a "source-at" special form.
For example,
(source-at "foo.scm" 10 2 ;; file line column ((source-at "foo.scm" 10 8 cons) (source-at "foo.scm" 10 8 x) (source-at "foo.scm" 10 10 y)))
would be equivalent to
(cons x y)
but with explicit source location information.
Actually this source-at form can be easily defined as a macro give me a few minutes.
Marc
On May 31, 2013, at 9:40 AM, Marc Feeley feeley@IRO.UMontreal.CA wrote:
On May 31, 2013, at 9:28 AM, Nathan Sorenson takeoutweight@hotmail.com wrote:
When translating Clojure code to Scheme, I'm finding it quite easy to preserve source code information for the Gambit interpreter, so that when I get exceptions I am notified of the offending Clojure source lines. Is there an obvious way I could do this for compiled code as well?
There isn't a form for this, but that would be useful and I have thought about adding it. Something like a "source-at" special form.
For example,
(source-at "foo.scm" 10 2 ;; file line column ((source-at "foo.scm" 10 8 cons) (source-at "foo.scm" 10 8 x) (source-at "foo.scm" 10 10 y)))
would be equivalent to
(cons x y)
but with explicit source location information.
Actually this source-at form can be easily defined as a macro give me a few minutes.
Here's the implementation of source-at.
Marc
;;; File: source-at.scm
(##define-syntax source-at (lambda (form)
(define (unwrap x) (if (##source? x) (##source-code x) x))
(apply (lambda (_ path line col expr) (##make-source (unwrap expr) (##make-locat (##path->container (unwrap path)) (##make-filepos (- (unwrap line) 1) (- (unwrap col) 1) 0)))) (unwrap form))))
;; test it:
(define (f x y) (+ 1 (source-at "foo.scm" 10 2 ;; file line column (/ x y))))
(pp (f 100 0))
;; Output: ;; ;; % gsi source-at.scm ;; *** ERROR IN f, "foo.scm"@10.2 -- Divide by zero ;; (/ 100 0)
Just for clarity does this mean Gambit's eval and compiler not only takes a sexp input (with or without location info annotations, depending on specific route), but also takes some kind of source code special object tree format?
2013/5/31 Marc Feeley feeley@iro.umontreal.ca
On May 31, 2013, at 9:40 AM, Marc Feeley feeley@IRO.UMontreal.CA wrote:
On May 31, 2013, at 9:28 AM, Nathan Sorenson takeoutweight@hotmail.com
wrote:
When translating Clojure code to Scheme, I'm finding it quite easy to
preserve source code information for the Gambit interpreter, so that when I get exceptions I am notified of the offending Clojure source lines. Is there an obvious way I could do this for compiled code as well?
There isn't a form for this, but that would be useful and I have thought
about adding it. Something like a "source-at" special form.
For example,
(source-at "foo.scm" 10 2 ;; file line column ((source-at "foo.scm" 10 8 cons) (source-at "foo.scm" 10 8 x) (source-at "foo.scm" 10 10 y)))
would be equivalent to
(cons x y)
but with explicit source location information.
Actually this source-at form can be easily defined as a macro give me a
few minutes.
Here's the implementation of source-at.
Marc
;;; File: source-at.scm
(##define-syntax source-at (lambda (form)
(define (unwrap x) (if (##source? x) (##source-code x) x)) (apply (lambda (_ path line col expr) (##make-source (unwrap expr) (##make-locat (##path->container (unwrap path)) (##make-filepos (- (unwrap line) 1) (- (unwrap col) 1) 0)))) (unwrap form))));; test it:
(define (f x y) (+ 1 (source-at "foo.scm" 10 2 ;; file line column (/ x y))))
(pp (f 100 0))
;; Output: ;; ;; % gsi source-at.scm ;; *** ERROR IN f, "foo.scm"@10.2 -- Divide by zero ;; (/ 100 0)
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On 2013-06-01, at 2:42 AM, Mikael mikael.rcv@gmail.com wrote:
Just for clarity does this mean Gambit's eval and compiler not only takes a sexp input (with or without location info annotations, depending on specific route), but also takes some kind of source code special object tree format?
Yes that is correct. They handle plain s-expressions and also s-expressions wrapped with "source" objects (created with ##make-source) that contain source location information.
Marc
(define (f x y) (+ 1 (source-at "foo.scm" 10 2 ;; file line column (/ x y))))
This is a clean approach, however one would have to be careful not to wrap macro symbols this way or else one would cause a "can't take the value of a macro" error, no?
I considered only applying the macro to lists but unbound variable errors won't report correct line info in that case. Another option would be to maintain a list of scheme macro symbols and ensure the clojure-scheme compiler doesn't attach source info to them. Even though it's now possible to run the compiler on Gambit, I want to maintain the possibility of doing the compilation entirely within the JVM, so I won't have any way to resolve symbols at run-time to see if they are macros.
Perhaps an easier approach would be to use a reader-macro, so that source info can be applied before regular macro expansion. Is it possible to tell gsc to use a custom read table? Or perhaps the ast parser would clobber the line info produced by any reader macros?
On Jun 1, 2013, at 10:54 PM, Nathan Sorenson takeoutweight@hotmail.com wrote:
(define (f x y) (+ 1 (source-at "foo.scm" 10 2 ;; file line column (/ x y))))
This is a clean approach, however one would have to be careful not to wrap macro symbols this way or else one would cause a "can't take the value of a macro" error, no?
I don't think so because the interpreter and compiler unwrap symbols when checking if they refer to macros. Do you have an example where this is not happening?
I considered only applying the macro to lists but unbound variable errors won't report correct line info in that case.
That is a bug in the interpreter that was introduced not too long ago (just type an undefined variable at the REPL and the error will be reported without location information). It is now fixed in the Gambit repo.
Another option would be to maintain a list of scheme macro symbols and ensure the clojure-scheme compiler doesn't attach source info to them. Even though it's now possible to run the compiler on Gambit, I want to maintain the possibility of doing the compilation entirely within the JVM, so I won't have any way to resolve symbols at run-time to see if they are macros.
Perhaps an easier approach would be to use a reader-macro, so that source info can be applied before regular macro expansion. Is it possible to tell gsc to use a custom read table? Or perhaps the ast parser would clobber the line info produced by any reader macros?
I think this is not relevant. Can you double check?
Marc
I don't think so because the interpreter and compiler unwrap symbols when checking if they refer to macros. Do you have an example where this is not happening?
This demonstrates the problem I'm having in using a macro to attach source info to symbols that name macros:
;OK (eval `(#(,##source2-marker lambda "foo" 1) () 'hi))
;Also OK (eval `(,(source-at "foo" 1 2 'lambda) () 'hi))
;fails (eval '((source-at "foo" 1 2 lambda) () 'hi)) *** ERROR IN "foo.cljscm"@1.2 -- Macro name can't be used as a variable: lambda
;likewise when compiling: ;;;; fail.scm ;;;;;; (define fail ((source-at "foo" 1 2 lambda) () 'hi))
$ gsc -debug fail.scm *** ERROR IN "/tmp/foo"@1.2 -- Macro name can't be used as a variable: lambda
I think the unquote in the two OK examples suggests that some kind of pre-macro-expansion evaluation stage is needed for attaching source information to symbols that refer to macros.
On 2013-06-02, at 1:02 AM, Nathan Sorenson nathan.d.sorenson@gmail.com wrote:
I don't think so because the interpreter and compiler unwrap symbols when checking if they refer to macros. Do you have an example where this is not happening?
This demonstrates the problem I'm having in using a macro to attach source info to symbols that name macros:
;OK (eval `(#(,##source2-marker lambda "foo" 1) () 'hi))
;Also OK (eval `(,(source-at "foo" 1 2 'lambda) () 'hi))
;fails (eval '((source-at "foo" 1 2 lambda) () 'hi)) *** ERROR IN "foo.cljscm"@1.2 -- Macro name can't be used as a variable: lambda
;likewise when compiling: ;;;; fail.scm ;;;;;; (define fail ((source-at "foo" 1 2 lambda) () 'hi))
$ gsc -debug fail.scm *** ERROR IN "/tmp/foo"@1.2 -- Macro name can't be used as a variable: lambda
I think the unquote in the two OK examples suggests that some kind of pre-macro-expansion evaluation stage is needed for attaching source information to symbols that refer to macros.
Thanks for clarifying. Now I see what the issue is. It is due to Gambit's macro expansion algorithm which does not recursively expand the head of a form before determining if it is a special form.
There are a few solutions I can think of. But let me ask this first, can't you simply avoid wrapping the head of special forms and macros? That location information will be (mostly) ignored anyway. What matters is the location of the complete expression. In other words the code generated by your compiler for (lambda (x) x) should look like:
(source-at "foo" 1 1 (lambda (x) (source-at "foo" 1 13 x))
instead of
((source-at "foo" 1 2 lambda) (x) (source-at "foo" 1 13 x))
or
((source-at "foo" 1 2 lambda) (source-at 1 9 (x)) (source-at "foo" 1 13 x))
If you can't help wrap the heads of special forms, then the source-at macro could recursively traverse the wrapped expression to process all the nested source-at forms.
Marc
There are a few solutions I can think of. But let me ask this first, can't you simply avoid wrapping the head of special forms and macros? That location information will be (mostly) ignored anyway. What matters is the location of the complete expression.
Yes, avoiding wrapping macro symbols is definitely one option. I would like to preserve the ability to do the compilation on the Java VM Clojure side, so I would have to maintain a blacklist of scheme macro symbols and explicitly avoid adding source info to them. I would not be able to introspect gambit's compile-time environment and see which symbols named macros.
One complication is that I provide an "in-line scheme" macro, scm*, for Clojure code, (similar to c-lambda for in-line C). This would enable users to define and use arbitrary scheme macros, so my macro blacklist would not work here; users might define and use new scheme macros that aren't on the blacklist. I could avoid attaching source macros to any head-position symbols generated via scm*, I suppose... Maybe I will do this for now.
It would be nice if clojure-scheme could indiscriminately tag every form without foreknowledge of what the symbol names. That's why my first impulse was reader evaluation of some sort -- it simplifies the code generation process.
On 2013-06-02, at 11:06 AM, Nathan Sorenson takeoutweight@hotmail.com wrote:
There are a few solutions I can think of. But let me ask this first, can't you simply avoid wrapping the head of special forms and macros? That location information will be (mostly) ignored anyway. What matters is the location of the complete expression.
Yes, avoiding wrapping macro symbols is definitely one option. I would like to preserve the ability to do the compilation on the Java VM Clojure side, so I would have to maintain a blacklist of scheme macro symbols and explicitly avoid adding source info to them. I would not be able to introspect gambit's compile-time environment and see which symbols named macros.
One complication is that I provide an "in-line scheme" macro, scm*, for Clojure code, (similar to c-lambda for in-line C). This would enable users to define and use arbitrary scheme macros, so my macro blacklist would not work here; users might define and use new scheme macros that aren't on the blacklist. I could avoid attaching source macros to any head-position symbols generated via scm*, I suppose... Maybe I will do this for now.
It would be nice if clojure-scheme could indiscriminately tag every form without foreknowledge of what the symbol names. That's why my first impulse was reader evaluation of some sort -- it simplifies the code generation process.
In that case the simplest solution I see is for the source-at macro to recursively traverse its subform. That way subforms that are not expressions will also get processed correctly. See the attached code.
As you say the reader could also be doing this job and I think this would be cleaner. However it would be necessary to invent a new syntactic form for this and I don't yet have a bright idea. Alternatively, you could enable the #. reader macro, that allows read-time evaluation, and generate a #. that explicitly tags the expression.
Marc
;;; File: source-at.scm
(##define-syntax source-at (lambda (source-at-form)
(define (unwrap x) (if (##source? x) (##source-code x) x))
(define (wrap path line col code) (##make-source code (##make-locat (##path->container path) (##make-filepos (- line 1) (- col 1) 0))))
(define (rewrap-list code) (cond ((null? code) code) ((pair? code) (cons (rewrap (car code)) (rewrap-list (cdr code)))) (else (rewrap code))))
(define (rewrap form)
(define (return code) (if (##source? form) (##make-source code (##source-locat form)) code))
(let ((code (unwrap form))) (cond ((pair? code) (if (eq? (unwrap (car code)) 'source-at)
(rewrap (apply (lambda (_ path line col subform) (wrap (unwrap path) (unwrap line) (unwrap col) (unwrap (rewrap subform)))) code))
(return (rewrap-list code))))
((vector? code) (return (list->vector (map rewrap (vector->list code)))))
(else form))))
(rewrap source-at-form)))
;; test it:
(define (f x y) (cons 'hello (source-at "foo.scm" 10 1 ;; file line column ((source-at "foo.scm" 11 2 if) (source-at "foo.scm" 12 3 (< x 0)) (source-at "foo.scm" 13 4 ((source-at "foo.scm" 14 5 quote) (source-at "foo.scm" 15 6 #((source-at "foo.scm" 16 7 111) (source-at "foo.scm" 17 8 222))))) (source-at "foo.scm" 18 9 (/ x y))))))
(pp (f -1 2)) (pp (f 100 0))
;; Output: ;; ;; % gsi source-at.scm ;; (hello . #(111 222)) ;; *** ERROR IN f, "foo.scm"@18.9 -- Divide by zero ;; (/ 100 0)
Alternatively, you could enable the #. reader macro, that allows read-time evaluation, and generate a #. that explicitly tags the expression.
What's the best way to do this for gsc? I've tried using -e but it doesn't seem to affect the compiler's reader:
gsc -e "(readtable-eval-allowed?-set (current-readtable) #t)" test.scm
What's the best way to do this for gsc? I've tried using -e but it doesn't seem to affect the compiler's reader: gsc -e "(readtable-eval-allowed?-set (current-readtable) #t)" test.scm
Nevermind! I didn't realize the readtable-eval-allowed?-set didn't mutate the readtable. This seems to work:
gsc -e "(current-readtable (readtable-eval-allowed?-set (current-readtable) #t))" -exe test.scm
-
Marc Feeley -
Mikael -
Nathan Sorenson -
Nathan Sorenson