2013/11/6 Marc Feeley <feeley@iro.umontreal.ca>

Gambit's threading model will be two tiered. There's the concept of Gambit virtual machine (VM) and the concept of "processor". These conceptually correspond to the classical operating system abstractions of process and thread, but these abstractions are not a 1-to-1 mapping to the OS abstractions. In fact, each processor is an OS thread and a VM corresponds to a self contained address space.

So a VM has an independent set of global variables and a heap (where Scheme objects are allocated). A VM cannot access the global variables and the heap of another VM, at least not directly. Within a VM, there are multiple processors (OS threads) running. These processors can share objects and access the same global environment. So a VM is the natural choice for implementing Termite's concept of "process", whereas the shared-memory concurrency provided by processors is ideal for implementing futures.

Aha - this abstraction makes enormous sense.

Please reformulate your questions with this new information.

Am all clear now, thank you.

Marc

To your request for feedback regarding whether to change the linker data structures to record module dependencies,

I would propose you instead only implement a fundamental core functionality for the user to declare whether a linked-in module should also be executed on start.

This way, you leave dependency handling logics all to the user to perform himself programmatically, just as it is now - preliminarily I believe this would be for the win in the bigger picture, for instance considering the level of flexibility and control it gives.

This is my preliminary feedback.

Find my reasoning behind this below, and also four followup questions, highlighted.

Best regards,

Mikael

Context

So, the levels are

Gambit global level contains one or more Gambit VM:s.

A VM is/represents/has a Scheme environment (global variables environment, address space, etc).

Each Gambit VM contains one or more Gambit processors.

A Gambit processor is running at max one CPU core at a time, and generally Gambit processor creation implies an OS thread creation.

And, each Gambit processor is running one or more green threads right?

And, there is some way that the user can assign and reassign executing processor within the VM, for a green thread?

Ok context understood.

Problem

So the conversation topic now is, that now that C linking/loading means a concurrent loading to the C level of all Gambit VM:s at the same time, then how should injection and execution of the loaded Scheme code be done into the VM:s, now that the user wants differentiated behavior between VM:s.

So problems that come with this are that for C code [modules] loaded, on the one hand

* you need a way to define what code [modules] is actually executed in the primordial VM&processor as code not intended to be executed there on load can be linked in too, and, on the other hand

* you need a way to specify what code [modules] should be executed in other VM:s.

This is the same problem as Unix and other OS:es face on boot: where to start execution, what's the first process and how to commence operations from there. The difference is just cosmetic in that a Unix system has the modules (the "init" program etc. and library files) in a filesystem while Gambit has them also on the heap already (the prelinked modules that this conversation are about).

Indeed, in Unix each module (executable & library) has a dependencies definition, and the OS loader is tasked to loads those deps.

Possible solution (A): Gambit bundles dependency loading logic. |preload-module| + hook

Gambit can reuse this as |preload-module| (as you suggested) and due to its higher level of abstraction have such dep definitions not just per module but per lambda in a module (as you suggested).

For this to be fully satisfactory, (as you said) the user needs to be able to inject modules both in form of interpreted and compiled code on runtime.

This probably also means there needs to be some runtime hook for a module system to perform the actual dependency loading, at the most basic level meaning resolving what already-linked-in module is actually meant by a particular module name specified to |preload-module|.

Possible solution (B): Gambit does not bundle dependency loading logic. (declare (not execute-on-load)) / (load module #!optional (execute-on-load? #t)) , |create-vm!|, |inject-module!|

So, the alternative would be something like vyzo suggested above: That

Gambit not contains recursive dependency loading logic per the suggestion above, but just

1) a way to specify which linked-in modules should be actually executed by the primordial VM&processor on start

(all linked-in modules, executed on start or not, can be commanded to be executed by the interface described below)

, and then

2) a programmatic interface to handle process creation and module execution, including operations to

* create a Gambit VM including specifying the module it should be started (injected and executed on its start) with - |create-vm!| , and

* a command to inject & execute a given module into the current VM - |inject-module!|

..and like in the suggestion above, modules here can either be interpreted or compiled, and linked in or loaded on runtime.

1. could be achieved with an argument to |load| and for when |load| happens on executable start due to C linking, using a declare form, like, (declare (not execute-on-load)) .

Both (A) and (B) require some way to enumerate modules, be it as a symbol name or using a first-class object representation.

In this moment given my current understanding, I'd vote rather for (B).

(B) is more basic and fundamental and programmatic.

Also, (A) can be implemented in terms of (B) anyhow, I would believe, adequately well?

Please share

* if you see any options beyond these two, and

* if you see any reason for Gambit to do (A) that I may not have understood right now.