What you describe is a common problem when adding a REPL to a program in order to monitor it and debug it. You want the REPL to interact with the user independently from what the main program is doing. In other words, conceptually you want 2 threads: one running the main program, and one running the REPL.
To avoid synchronization problems and allow calls between C and Scheme using Gambit's C-interface, these conceptual threads should not be OS threads. It is best to use a cooperative thread model, where only one of these two conceptual threads is active at any given point, and control is passed explicitly between the "main thread" and the "REPL thread". I'll get to how this can be implemented in a moment.
Note that when a C to Scheme call returns (to C), or when Scheme code calls a C function, the state of the Scheme world is suspended (in other words all of the Scheme threads are blocked). It is only when C makes a call to a Scheme function, or when a Scheme to C call returns, that the Scheme thread scheduler gets a chance to multiplex the execution of the threads (the multiplexing is done at regular intervals using a "heartbeat" timer interrupt, but only when control is in the Scheme code). Starting a Scheme thread for the REPL is pointless if control (mostly) remains in the C world because it doesn't guarantee that the REPL thread actually runs.
Given that there are multiple Scheme threads alive in the Scheme world, you might wonder what happens when a C to Scheme function call is performed... in which Scheme thread is the Scheme function executed? It is simply the same Scheme thread that was executing when control returned to C from Scheme.
In your case, you probably need 2 Scheme threads. One is responsible for executing the REPL (call it thread R), and one is responsible for executing the Scheme functions that are called from C (call it thread S).
To implement this, the idea is to have the C code make calls to a Scheme "heartbeat" function in thread S on a regular basis. This is usually done in the application's main event loop. The heartbeat function then calls (##thread-heartbeat!) to allow the Scheme thread scheduler to run thread R (and possibly other Scheme threads that might have been created). After the Scheme scheduler has given a fraction of a second for each Scheme thread to advance their computation, the Scheme thread S will be resumed, and at the end of the heartbeat function control returns back to C. At the next C to Scheme call of the heartbeat function, the Scheme thread scheduler will give another fraction of a second of execution time to each Scheme thread, etc.
Please check the Gambit REPL example, in particular file intf.scm, which contains an improved heartbeat function which passes back to C the time that should elapse before the heartbeat function is called again (this period of time depends on information that the scheduler has on what the Scheme threads are doing, such as sleeping with a timeout, doing I/O, computing, etc). This reduces the power consumption.
Marc
On 2012-02-10, at 3:11 PM, François Magnan wrote:
Hi,
I am currently building an iOS app with Gambit and I (of course) need a REPL-server in it so I can develop interactively instead of recompiling the app every time I make a change. I saw the REPL-server that Marc developed in the iOS example folder of Gambit distribution so I tried to integrate it in my app. I was able to make it work but I have some problems.
When I first tried it I tried to run the repl server in a separate gambit thread. I therefore defined the following:
(c-define (init-repl-server) () void "initREPLServer" "" ; Start a REPL server (thread-start! (make-thread (lambda () (display "\n\nStarting REPL server on port 7000\n") (repl-server-start "toto")))))
In the objective-c method (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions I called the gambit setup and then the initREPLServer(); function.
This didn't work: when the C function returned the Gambit thread died (this is my interpretation) and I could not connect to the server.
I therefore tried put an infinite-loop at the end of the (init-repl-server) function. It worked but the app would not finish the startup process since "application didFinishLaunchingWithOptions" call never returns YES as expected.
I then tried to run the scheme program in a separate thread (in that thread the call to (init-repl-server) in C would not have to return and would keep running). [NSThread detachNewThreadSelector:@selector(initReplServer) toTarget:self withObject:nil]; I could then connect to the repl server but all assignments in objects are ignored by the interface loop since they are done in a separate thread. I then had the idea to use objective-c blocks (kind of closures in the objective-c world).
[[NSOperationQueue mainQueue] addOperationWithBlock:^{ object.slot=value; }];
This worked.... All the updates were now done in the GUI while working in the remote REPL. One day I tried to instantiate a UIWebView and the app crashed. UIWebView only works in the "main" thread.
I don't like the idea to have the REPL-server and Scheme program in a separate thread. So I tried to put just the REPL-server in a separate thread and run my actual scheme program in the main thread. In this setup I could connect to the remote REPL but all the variables in the Scheme world are not pointing to the right memory location so everything is scrambled.
Is there a way out of this...? Can my program run in the main thread while still having a working remote REPL to debug it?
Thank you, Francois Magnan
Gambit-iphone mailing list Gambit-iphone@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-iphone
Afficher les réponses par date
Hi,
Thank you very much Marc for the explanation. I was able to make it work with a "very simple" heartbeat function. The advanced heartbeat function in the iOS example is beyond my reach for the moment. In my case I need a REPL only for development purposes so I don't need to save power consumption by delaying calls to the heartbeat function.
Is the remote debugger API (that you can specify in the Gambit launch args) described somewhere? This would be my next step towards a fully functional development environment.
Thank you, Francois
On 2012-02-10, at 23:31, Marc Feeley wrote:
What you describe is a common problem when adding a REPL to a program in order to monitor it and debug it. You want the REPL to interact with the user independently from what the main program is doing. In other words, conceptually you want 2 threads: one running the main program, and one running the REPL.
To avoid synchronization problems and allow calls between C and Scheme using Gambit's C-interface, these conceptual threads should not be OS threads. It is best to use a cooperative thread model, where only one of these two conceptual threads is active at any given point, and control is passed explicitly between the "main thread" and the "REPL thread". I'll get to how this can be implemented in a moment.
Note that when a C to Scheme call returns (to C), or when Scheme code calls a C function, the state of the Scheme world is suspended (in other words all of the Scheme threads are blocked). It is only when C makes a call to a Scheme function, or when a Scheme to C call returns, that the Scheme thread scheduler gets a chance to multiplex the execution of the threads (the multiplexing is done at regular intervals using a "heartbeat" timer interrupt, but only when control is in the Scheme code). Starting a Scheme thread for the REPL is pointless if control (mostly) remains in the C world because it doesn't guarantee that the REPL thread actually runs.
Given that there are multiple Scheme threads alive in the Scheme world, you might wonder what happens when a C to Scheme function call is performed... in which Scheme thread is the Scheme function executed? It is simply the same Scheme thread that was executing when control returned to C from Scheme.
In your case, you probably need 2 Scheme threads. One is responsible for executing the REPL (call it thread R), and one is responsible for executing the Scheme functions that are called from C (call it thread S).
To implement this, the idea is to have the C code make calls to a Scheme "heartbeat" function in thread S on a regular basis. This is usually done in the application's main event loop. The heartbeat function then calls (##thread-heartbeat!) to allow the Scheme thread scheduler to run thread R (and possibly other Scheme threads that might have been created). After the Scheme scheduler has given a fraction of a second for each Scheme thread to advance their computation, the Scheme thread S will be resumed, and at the end of the heartbeat function control returns back to C. At the next C to Scheme call of the heartbeat function, the Scheme thread scheduler will give another fraction of a second of execution time to each Scheme thread, etc.
Please check the Gambit REPL example, in particular file intf.scm, which contains an improved heartbeat function which passes back to C the time that should elapse before the heartbeat function is called again (this period of time depends on information that the scheduler has on what the Scheme threads are doing, such as sleeping with a timeout, doing I/O, computing, etc). This reduces the power consumption.
Marc
On 2012-02-10, at 3:11 PM, François Magnan wrote:
Hi,
I am currently building an iOS app with Gambit and I (of course) need a REPL-server in it so I can develop interactively instead of recompiling the app every time I make a change. I saw the REPL-server that Marc developed in the iOS example folder of Gambit distribution so I tried to integrate it in my app. I was able to make it work but I have some problems.
When I first tried it I tried to run the repl server in a separate gambit thread. I therefore defined the following:
(c-define (init-repl-server) () void "initREPLServer" "" ; Start a REPL server (thread-start! (make-thread (lambda () (display "\n\nStarting REPL server on port 7000\n") (repl-server-start "toto")))))
In the objective-c method (BOOL)application:(UIApplication *)application didFinishLaunchingWithOptions:(NSDictionary *)launchOptions I called the gambit setup and then the initREPLServer(); function.
This didn't work: when the C function returned the Gambit thread died (this is my interpretation) and I could not connect to the server.
I therefore tried put an infinite-loop at the end of the (init-repl-server) function. It worked but the app would not finish the startup process since "application didFinishLaunchingWithOptions" call never returns YES as expected.
I then tried to run the scheme program in a separate thread (in that thread the call to (init-repl-server) in C would not have to return and would keep running). [NSThread detachNewThreadSelector:@selector(initReplServer) toTarget:self withObject:nil]; I could then connect to the repl server but all assignments in objects are ignored by the interface loop since they are done in a separate thread. I then had the idea to use objective-c blocks (kind of closures in the objective-c world).
[[NSOperationQueue mainQueue] addOperationWithBlock:^{ object.slot=value; }];
This worked.... All the updates were now done in the GUI while working in the remote REPL. One day I tried to instantiate a UIWebView and the app crashed. UIWebView only works in the "main" thread.
I don't like the idea to have the REPL-server and Scheme program in a separate thread. So I tried to put just the REPL-server in a separate thread and run my actual scheme program in the main thread. In this setup I could connect to the remote REPL but all the variables in the Scheme world are not pointing to the right memory location so everything is scrambled.
Is there a way out of this...? Can my program run in the main thread while still having a working remote REPL to debug it?
Thank you, Francois Magnan
Gambit-iphone mailing list Gambit-iphone@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-iphone
-
Francois Magnan -
Marc Feeley