I had to smile reading this. Well, one can consider it a challenge getting into understanding it :).

Some things just take time to find out (when I first looked at the code I did understand almost nothing; now I have a grasp of about a forth of the code or so I guess (memory layout, basic data accessor functions etc., intermediate code, stack/register functionality..; learning small bits every few days)).

I agree that I don't get quite many decisions. Some more documentation would definitely help. (I remember there is a paper from Marc about garbage collection; is there one about the Gambit intermediate code, or the stack/register machine workings?)

Today I looked into how object->u8vector works. This is one of the places where (in beta 21) many macros are being used.

* it's not particularly nice looking, but the convention to prefix most macros with macro- is actually helpful.

(* that many structures are being built with the macro: keyword, is of course for performance reasons; but then generating (by use of define-prim) functions for accessing the structures clutters the code quite a bit up, maybe it makes sense in places, maybe it cannot be done better; I'm actually even more of a code generation junkie I guess, I've already written about 3 structure generation libraries on top of Gambit's (lately I started to think about "finally writing" a functional update record library until I realized I already did one :).)

* in the case of object->u8vector, I think all those helper macros would better have been put into a separate file (one for fifos, for example), and just locally included.

I realize that object->u8vector has been changed between beta20(?) and beta21, there has been "termite specific serialization" code before. I was wondering what that has been. These are things where a public code versioning repository would be wonderful. Also for example to see what exactly changed regarding namespacing (because of snow) in beta 22 which broke chjmodule (I'll write a separate mail about this when I finally get to it).

One thing I still don't get are those define-prim which don't have a body, where is the code that's actually being executed for them?

I would suggest using GIT as code versioning repository one has not already been decided upon, I'm using it for about a year now and I really like it (I've even started writing a Gambit library for accessing the git tools, what's there already is a parser of git-log output and I'm just in the midst of developing a better git-bisect algorithm with it).

The best thing for learning how Gambit internals work (or at least it's runtime library part), would be to run Gambit itself in the interpreter (possibly even with tail calls disabled), and/or at least compile it in safe mode. Maybe Gambit could also be split into a smaller core and libraries for making interpreted loading of the non-core parts possible?

One thing I'm always thinking about when you suggest some compiler expansion to Marc: with a good enough module system those could just be imported as functions from a module, with some hint to the module system to inline the definition into the client module. This would also allow the user to decide (by choosing his imports (or import parameters) wisely) whether he wants his object files to become bigger because of a bigger expansion, or rather stay with slower cross module calls but getting smaller object files. (BTW today I've measured that cross module ("cross block", rather?) function calls are only taking about 120 cycles on beta 22; I remember them having been about 200 or more in the past, am I seeing a nice improvement here? BTW nowadays I'm using an inline macro for just inlining costly calls across modules, so no such big deal anyway :).)

The reason for me looking at object->u8vector has been that I wondered why it would allocate so much memory. I'm suspecting there is room for quite some speedup (I'm writing an object database, and am pondering about the best way for object serialization/compression, and have been quite impressed about how compact the output of the Gambit serializer is, but it's rather too slow for my purposes). But I'm wondering how to best go over improving it: recompiling/linking Gambit after each change attempt doesn't sound very productive, especially when I'll just get a segfault most of the time at first (and not very useful backtraces in gdb)--that's where safe internals would help; building a new variant as chjmodule has the problem of all those macros which I don't have when not including most of those header files, which I'm not sure how well I could do that. So most times when I come to reimplement something like this is to just rewrite it without requiring those macros/headers. But then it's a chjmodule and could not just be merged back into Gambit straight away (and I always fear it breaks when Gambit internals change). That's why I think modularising Gambit would be a good thing.

I think Snow is probably good for sharing modules with other systems, but maybe less so for lowlevel integrated stuff like Gambit's own parts, or much of what I'm doing with chjmodules (which is also much experimentation). This is probably an ongoing search for me for the next few months.

PS. macro-character-port-rlo is being defined in _io#.scm:

(define-type-of-port character-port ... macros: ;; <--why they are macros which I cannot use from chjmodule :) prefix: macro- ... rlo ; low pointer (start of unread characters)

thus it's an accessor macro for the character-port data structure.

Christian.

Bradley Lucier wrote:

Marc:

Your runtime is a masterful lesson on how to write code that writes code that writes code. But it's nearly totally undocumented, and if you really want to turn gambit into an open-source project you need to give out more information on what the code does and how it is changing.

Brad _______________________________________________ Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list