[gambit-list] Gambit linking

Mikael mikael.rcv at gmail.com
Wed Nov 6 13:30:06 EST 2013


2013/11/6 Marc Feeley <feeley at iro.umontreal.ca>
>
> Gambit's threading model will be two tiered.  There's the concept of
> Gambit virtual machine (VM) and the concept of "processor".  These
> conceptually correspond to the classical operating system abstractions of
> process and thread, but these abstractions are not a 1-to-1 mapping to the
> OS abstractions.  In fact, each processor is an OS thread and a VM
> corresponds to a self contained address space.
>
> So a VM has an independent set of global variables and a heap (where
> Scheme objects are allocated).  A VM cannot access the global variables and
> the heap of another VM, at least not directly.  Within a VM, there are
> multiple processors (OS threads) running.  These processors can share
> objects and access the same global environment.  So a VM is the natural
> choice for implementing Termite's concept of "process", whereas the
> shared-memory concurrency provided by processors is ideal for implementing
> futures.
>

Aha - this abstraction makes enormous sense.


> Please reformulate your questions with this new information.
>

Am all clear now, thank you.


> Marc
>


To your request for feedback regarding whether to change the linker data
structures to record module dependencies,

I would propose you instead only implement a fundamental core functionality
for the user to declare whether a linked-in module should also be executed
on start.

This way, you leave dependency handling logics all to the user to perform
himself programmatically, just as it is now - preliminarily I believe this
would be for the win in the bigger picture, for instance considering the level
of flexibility and control it gives.


This is my preliminary feedback.


Find my reasoning behind this below, and also four followup questions,
highlighted.


Best regards,
Mikael

 -



*Context*
So, the levels are

Gambit global level contains one or more Gambit VM:s.
A VM is/represents/has a Scheme environment (global variables environment,
address space, etc).
Each Gambit VM contains one or more Gambit processors.
A Gambit processor is running at max one CPU core at a time, and generally
Gambit processor creation implies an OS thread creation.

And, each Gambit processor is running one or more green threads right?
And, there is some way that the user can assign and reassign executing
processor within the VM, for a green thread?


Ok context understood.


*Problem*
So the conversation topic now is, that now that C linking/loading means a
concurrent loading to the C level of all Gambit VM:s at the same time, then
how should injection and execution of the loaded Scheme code be done into
the VM:s, now that the user wants differentiated behavior between VM:s.

So problems that come with this are that for C code [modules] loaded, on
the one hand
 * you need a way to define what code [modules] is actually executed in the
primordial VM&processor as code not intended to be executed there on load
can be linked in too, and, on the other hand
 * you need a way to specify what code [modules] should be executed in
other VM:s.

This is the same problem as Unix and other OS:es face on boot: where to
start execution, what's the first process and how to commence operations
from there. The difference is just cosmetic in that a Unix system has the
modules (the "init" program etc. and library files) in a filesystem while
Gambit has them also on the heap already (the prelinked modules that this
conversation are about).

Indeed, in Unix each module (executable & library) has a dependencies
definition, and the OS loader is tasked to loads those deps.

*Possible solution (A): Gambit bundles dependency loading logic.
|preload-module| + hook*
Gambit can reuse this as |preload-module| (as you suggested) and due to its
higher level of abstraction have such dep definitions not just per module
but per lambda in a module (as you suggested).

For this to be fully satisfactory, (as you said) the user needs to be able
to inject modules both in form of interpreted and compiled code on runtime.

This probably also means there needs to be some runtime hook for a module
system to perform the actual dependency loading, at the most basic level
meaning resolving what already-linked-in module is actually meant by a
particular module name specified to |preload-module|.


*Possible solution (B): Gambit does not bundle dependency loading logic.
(declare (not execute-on-load)) / (load module #!optional (execute-on-load?
#t)) , |create-vm!|, |inject-module!|*
So, the alternative would be something like vyzo suggested above: That

Gambit not contains recursive dependency loading logic per the suggestion
above, but just

 1) a way to specify which linked-in modules should be actually executed by
the primordial VM&processor on start
     (all linked-in modules, executed on start or not, can be commanded to
be executed by the interface described below)
      , and then

 2) a programmatic interface to handle process creation and module
execution, including operations to

      * create a Gambit VM including specifying the module it should be
started (injected and executed on its start) with - |create-vm!| , and

      * a command to inject & execute a given module into the current VM -
|inject-module!|

..and like in the suggestion above, modules here can either be interpreted
or compiled, and linked in or loaded on runtime.

1. could be achieved with an argument to |load| and for when |load| happens
on executable start due to C linking, using a declare form, like, (declare
(not execute-on-load)) .



Both (A) and (B) require some way to enumerate modules, be it as a symbol
name or using a first-class object representation.




In this moment given my current understanding, I'd vote rather for (B).

(B) is more basic and fundamental and programmatic.

Also, (A) can be implemented in terms of (B) anyhow, I would believe,
adequately well?


Please share

 * if you see any options beyond these two, and

 * if you see any reason for Gambit to do (A) that I may not have
understood right now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20131106/890fa982/attachment.htm>


More information about the Gambit-list mailing list