2013/9/27 Marc Feeley <feeley@iro.umontreal.ca>

On 2013-09-27, at 3:48 PM, Mikael <mikael.rcv@gmail.com> wrote:

> Dear Marc,
>
> When (load):ing a binary file ("myfile.o1" etc.) , would there be any way to do identifier renaming at load time?
>
> E.g.,
>
> echo "(define x 5)" >> myfile.scm
>
> gsc
>
> (compile-file "myfile.scm")
>
> (set! ##load-symbol-import (lambda (loaded-symbol-name) 'y))
>
> (load "myfile.o1")
>
> y ; => 5
>
> I believe this could be very useful for a module system - not sure though would be happy to understand if there's any problem about it. Without or with modification to Gambit's source.
>
> Best regards,
> Mikael
>

Are you thinking about this in order to load a module multiple times with different renamings?

Nope.

 

 That will not work because the C names will clash.

Noted.

 

Why don't you do the renaming at compile time?

Compilation takes time and at least in this moment I preliminarily understand that it would be very useful for a module system to be able to determine which identifiers a (load) will occupy, at the time of load rather than at the time of compile.

Compilation requires a special computing environment that may not be available at load time, and also takes time and other resources, so in using a module system, if you can skip a round of recompilation, you do it.

Renaming at load time would help the module system solve any potential namespace clashes between compiled modules at load time.

A worstcase scenario, I guess, would be that a user has two modules by the name X in different directories, that were compiled completely separately, and both export an identifier Y, so both's compiled object file contain an identifier Y or X#Y or alike that is the same in both modules.

Of course depending on what identifier naming scheme the module system has, identifier clashes between modules can happen anytime - it is reasonable to understand that a module system may want to import two compiled modules, that were each compiled in module system executions that were done in full isolation from each other (i.e. without access to one and the same "ns.dat" or alike file at the time of compile, for instance because of restrictive file privileges, or that compilation was done on different machines), making clashes expectable or at least possible depending on naming scheme design.

In this case, it would be extremely useful if the module system would be able to load Y for the two modules into different identifiers: X1#Y or alike for the first module and X2#Y or alike for the second module.

This loading into X1#Y & X2#Y etc. is not only about making the respective identifier get a globally unique name - if that was the purpose, a module system could just use random 128bit UUID:s for actual identifiers and thus omit clashes that way - beyond having a unique name globally, the identifiers loaded by Gambit are the ones used in (pp) and interactive debugging output and related structures, and this kind of renaming at load time would be the only way to do renaming that is effective also there and thus providing meaningful debugging during module system use (patching pp and all the interactive debugging code with a live renaming feature in their output logics would add much more complexity than doing the renaming at load time).

This idea has come to me from pondering on Per's reflections from making BH as well as giving some thought to what Gambit would need to be able to deliver an as complete/profound degree of module system integration as Chez and Racked have - spontaneously I would guess that within the context of using dynamically loadable C-compiled library/object files, the ability to rename identifiers to ensure uniqueness at load time would provide a complete/nonleaking identifier handling abstraction.

- While a module system in the absence of this needs a global state for ensuring that two different module files are never compiled to use one and the same identifier, and if this can't be achieved for instance due to file privilege reasons, need could arise to do almost-unlimited amounts of recompile work pretty much anytime.

(

Only for illustration, a lengthy example of when ability to rename identifiers on load time would be of use:

 * The machine A has the module system installed, and the admin user installs and compiles the SRFI:s for all to use.

   The admin's SRFI 13 gets the srfi13# namespace.

 * User B installs a local version of an SRFI (e.g. 13) and compiles and imports it.

   B's namespace handling gets readonly access to the admin's namespace database so it knows srfi13# is already occupied and goes with srfi13_2# .

 * Admin installs a new set of SRFI:s, say, an upgrade, and compiles it.

   The admin's namespace database knows the admin used srfi13# already however doesn't keep track of B's namespace use as B only had readonly access to, so the srfi13_2# namespace is chosen.

 * Admin changes package directory for all users globally to the new directory.

 * B imports a code project that uses the admin's SRFI 14 (which has the admin's SRFI 13 as a dependency) and then his own SRFI 13

   => clash! - both the admin's and B's SRFI 13 use the srfi13_2# namespace.

   In this basic example, the module system would just need to figure out correctly to recompile B's SRFI 13.

   Much much more complex usecases are possible though, where file privileges and alike are really against you so to speak, that could require enormous amounts of modules to be recompiled, causing a simple import to take, say, an hour, and require RAM quotas that are not available etc., making module import impossible.

)

Of course it can be a requirement on the renaming mechanism that it is required to rename to unique identifiers - this is a reasonable requirement on it really.

Thoughts/feedback/reflections, and, can this renaming at load time be done?

 

Marc

Mikael