Gorgeous. I was already setting the runtime options -:dar. The exception handler was the problem; because it's not running on the primordial thread I need to explicitly call the repl exception handler in the repl thread.

Basically I wrap an exception handler around the call to ##repl-debug-main which manually fires off ##repl-exception-handler-hook. If anyone knows a better way to do this please enlighten me.

The remote node (on which the repl worker for the local node is running) is sitting at a repl in the primordial thread itself. I'm a little curious: before I added my exception handler, why didn't the primordial thread on the remote node catch the exception and throw it into a sub-repl?

I have attached the new version which works great. I also optimized the remote reading and writing; previously it was sending and receiving every char as a message, which was dreadfully slow. Now it batches it up where possible. Reading from the repl was a little tricky - we don't want to poll, but we also want to read as many characters at once where possible. The current implementation sends off the characters if nothing is written after .1 seconds.

Also, in order to handle both the ",q" command and any possibility of the remote node crashing/quitting, I had to add a heartbeat to the repl worker process. The only other way I can think of handling quitting is if there's some cleanup function that is called when the primordial thread is terminated - I doubt it though. The heartbeat method isn't too bad - since it handles the worse case scenario of a hard crash.

Thanks again -

;;;;; --- REMOTE REPL ---

(define (start-repl-services node repl-input-proc-port repl-output-proc-port repl-worker-port)

;; Handles printing out to the remote repl (define output-handler (spawn (lambda ()

;; Block until a character is available. ;; When one is available, continually read ;; until nothing is written for .1 seconds, send ;; off the buffer for output, and start over. (let loop1 () (peek-char repl-worker-port) (input-port-timeout-set! repl-worker-port .1) (proc-port-display (read-all repl-worker-port read-char) repl-output-proc-port) (input-port-timeout-set! repl-worker-port +inf.0) (loop1)))))

;; Service to tell the remote repl we're alive (define repl-worker-heartbeat (spawn (lambda () (let loop () (thread-sleep! 1) (! node 'heartbeat) (loop)))))

;; Main loop - the input (spawn (lambda () (let loop () (let ((cmd (proc-port-read-line repl-input-proc-port))) (display cmd repl-worker-port) (write-char #\return repl-worker-port) (write-char #\linefeed repl-worker-port) (force-output repl-worker-port)

(if (equal? cmd ",q") (begin (terminate! output-handler) (terminate! repl-worker-heartbeat)) (loop)))))))

(define (make-repl node) (let ((op (current-output-proc-port)) (ip (current-input-proc-port)) (repl (self))) (remote-spawn node (lambda () (receive (pipe1 pipe2) (open-string-pipe)

(start-repl-services repl ip op pipe1) (set! ##thread-make-repl-channel (lambda () (##make-repl-channel-ports pipe2 pipe2)))

;; Fire up the repl with an explicit repl exception handler (with-exception-handler (lambda (exc) (##repl-exception-handler-hook exc ##thread-end-with-uncaught-exception!)) (lambda () (display "Firing up repl...\n") (##repl-debug-main)))))))

(let loop () (recv (('heartbeat) (loop)) (after 2 (display "\n------ Remote REPL closed -------- \n")))))

On Dec 4, 2007 7:11 AM, Christian Jaeger christian@pflanze.mine.nu wrote:

PS. see also the -:da flag of the Gambit system. Maybe it's enough if you somehow start up your remote nodes with that flag.

$ gsi -:da Gambit v4.0.1

...

(define t (thread-start! (make-thread (lambda() (error "hello") (+ 3

4)))))

...

(thread-join! t)

------------- REPL is now in #<thread #2> ------------- *** ERROR IN #<procedure #3>, (console)@1.49 -- hello

...

,b

0 #<procedure #3> (console)@1:49 (error "hello") 1 ##thread-start-action!

...

,y

0 #<procedure #3> (console)@1.49 (error "hello")

...

,(c 3)

7 ------------- REPL is now in #<thread #1 primordial> -------------

...

*** EOF again to exit $ gsi Gambit v4.0.1

...

(define t (thread-start! (make-thread (lambda() (error "hello") (+ 3

4)))))

...

(thread-join! t)

*** ERROR IN (console)@2.1 -- Uncaught exception: #<error-exception #2> (thread-join! '#<thread #3>) 1>

James Long wrote:

One last question. I'd like to get interrupts working, and anything ctrl related (like C-d). I think I can figure out how to implement remote interrupts (as long as I can fire off an interrupt in code), but I'm not sure how to capture the C-d behavior. The repl is run in a normal OS X terminal. Any tips?

You can catch Ctl-c (SIGINT interrupts) by setting current-user-interrupt-handler, and you can catch other interrupts by following the advice in https://webmail.iro.umontreal.ca/pipermail/gambit-list/2006-January/000529.h... (following that mail, I wrote an "interrupts" chjmodule, ask me for it if you're interested).

I'm not sure what to suggest for interrupting the remote side, though; if it should be a real signal, you could of course use kill(2), from another process on the same machine. If you have multiple scheme threads running on the remote node, that's probably not very convenient, though. So maybe you could instead run a separate scheme thread on the remote side, to which you can send a request to pseudo-interrupt the thread in question by throwing an interrupt exception in the context of the latter (I haven't done this yet, but you could take a look at how the default user-interrupt-handler is written; statprof ("statistical profiler" from Guillaume Germain, http://www.iro.umontreal.ca/~germaing/statprof.html) might also be instructive as it is getting at the continuation of the currently running thread (you want the continuation of a specific thread instead, and then I hope that by calling ##continuation-graft-with-winding you could throw an exception in the context of that thread (not sure, though)). I'm sure Marc can give you more concrete advice. Note that pure Scheme-level solutions won't interrupt C code, for that only a real signal can help, so to make it work really well, a two level approach would be useful, first try the scheme level, then after a timeout run kill(2). (For kill (and fork etc.) my cj-posix scheme module may be helpful (the last version, ask me for the current version if you want.)

Christian.

Christian Jaeger wrote:

I hope that by calling ##continuation-graft-with-winding you could throw an exception in the context of that thread (not sure, though)).

Ok, it didn't let me rest and I tested it out. Seems my hope is void: you can call a continuation from a foreign thread, but it is then not executed in the original thread, but in the current thread instead (the original thread continues normal execution). That makes sense, of course, since that allows one to transfer 'agents' to other threads (just what Termite is doing).

So you have to find out how to evaluate code in the context of another thread (I'd examine the code implementing the default signal handler), or let Marc tell you.

Christian.

(define cont #f);; note: this is to hold a raw continuation datastructure, not wrapped in a closure

(define-macro (%thread . body) `(thread-start! (make-thread (lambda () ,@body))))

(define t1 (%thread (let ((t (current-thread))) (println "thread t1, " t ", started") (##continuation-capture (lambda (c) (set! cont c) (let lp () (thread-sleep! 2) (println "thread " (current-thread) " is awake") (lp)))) (println "thread t1, " t " (" (current-thread) ") ending"))))

(define t2 (%thread (let ((t (current-thread))) (println "thread t2, " t ", started") (thread-sleep! 5) (##continuation-graft ;; hm, ##continuation-graft-with-winding does not exist anymore in Gambit v4.0.1 cont (lambda () ;;(error "an error from t2 in the context of t1?") "a value")) (println "thread t2, " t " (" (current-thread) ") ending"))))

with the error statement uncommented:

(load "threadcont")

"/tmp/chris/threadcont.scm"

thread t1, #<thread #2>, started

thread t2, #<thread #3>, started thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake (thread-join! t2) ;; returns immediately *** ERROR IN (console)@2.1 -- Uncaught exception: #<error-exception #4> (thread-join! '#<thread #3>) 1> thread #<thread #2> is awake thread #<thread #2> is awake (thread-join! t1) ;; 'hangs' thread #<thread #2> is awake thread #<thread #2> is awake <---(hitting ctl-c) ------------- REPL is now in #<thread #2> ------------- *** INTERRUPTED IN ##thread-sleep!

with the "a value" return:

(load "threadcont")

"/tmp/chris/threadcont.scm"

thread t1, #<thread #2>, started

thread t2, #<thread #3>, started thread #<thread #2> is awake thread #<thread #2> is awake thread t1, #<thread #2> (#<thread #3>) ending thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake (thread-join! t2) ;; returns immediately

thread #<thread #2> is awake

thread #<thread #2> is awake (thread-join! t1) ;; 'hangs' thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake thread #<thread #2> is awake <---- (hitting ctl-c) ------------- REPL is now in #<thread #2> ------------- *** INTERRUPTED IN ##thread-sleep!

(thread-join! t2) ;; the void value, of course, since "a value" is lost and the result of the "ending" println returned

Nevermind, that won't work for interrupts. I just thought about it a little more. The repl would never read from the string port if a command goes crazy and starts thrashing. Hm, we do need a real interrupt somehow, but I'm not sure if that's possible without disturbing the primordial thread.

So we do need something akin to your idea, which will only work if Gambit supports some kind of thread interrupting.

On Dec 5, 2007 6:26 PM, James Long longster@gmail.com wrote:

Interesting stuff Christian. The idea of executing something in the context of another thread is a bit competing though, right? A thread itself defines only one path of execution, so unless you mean to somehow stop a thread and replace its continuation with another one, I can't really imagine anything else. And doing that requires you to save the old continuation and make sure you set everything back up right.

It seems more natural to try and send a message to the thread. In this context, it's a repl service, so I figured we could try to communicate over the repl service string port. (NOTE: I am now calling the local process which reads commands and outputs the result a 'repl', and the remote process which evaluates the commands a 'repl service').

To achieve C-d, I replaced the line

(display cmd repl-worker-port)

with

(if (eof-object? cmd) (display "(raise 'eof)" repl-worker-port) (display cmd repl-worker-port))

and then my exception handler becomes

       ;; Fire up the repl with an explicit repl exception handler
       (with-exception-handler
        (lambda (exc)
          (if (symbol? exc)
              (cond ((eq? exc 'eof) (cmd-d)))
              (##repl-exception-handler-hook exc

##thread-end-with-uncaught-exception!))) (lambda () (##repl-debug-main)))))))

It worked pretty well actually, and you would do the same thing with interrupts. Now I just have to figure out how interrupts should actually behave... maybe C-c C-c should interrupt the local repl?

James

On Dec 5, 2007 9:08 AM, Christian Jaeger christian@pflanze.mine.nu wrote:

...

Christian Jaeger wrote:

...

I hope that by calling ##continuation-graft-with-winding you could throw an exception in the context of that thread (not sure, though)).

Ok, it didn't let me rest and I tested it out. Seems my hope is void: you can call a continuation from a foreign thread, but it is then not executed in the original thread, but in the current thread instead (the original thread continues normal execution). That makes sense, of course, since that allows one to transfer 'agents' to other threads (just what Termite is doing).

So you have to find out how to evaluate code in the context of another thread (I'd examine the code implementing the default signal handler), or let Marc tell you.

Christian.

-- James Long Coptix, Inc. longster@gmail.com

On Dec 5, 2007 6:08 AM, James Long longster@gmail.com wrote:

One last question. I'd like to get interrupts working, and anything ctrl related (like C-d).

IIRC, using bash, Ctrl-d doesn't generate a signal, just an EOF on stdin.