Re: [Tachyon-list] Inspiration: Self Klein's Metacircular Virtual Machine Kit
Oups, did a private reply to mark. Mark is there a way to set the "reply-to" field of the mailing list to the mailing list itself? Thanks -------- Message original -------- Sujet: Re: [Tachyon-list] Inspiration: Self Klein's Metacircular Virtual Machine Kit Date : Thu, 29 Apr 2010 23:48:05 -0400 De : Erick Lavoie <erick.lavoie@gmail.com> Pour : Marc Feeley <feeley@iro.umontreal.ca> There is still a couple of things that worries me. Given that they were mostly three working on Klein and that they never achieve a level of performance similar to their C++ implementation, I feel that the problem of boostrapping a system efficiently (in terms of development time) to achieve both simplicity and performance is not completely understood and therefore risky. What I get from the paper, is that tools for inspecting and debugging the VM are of paramount importance but can be really tricky to get right. That's something we should take into account in planning the work ahead. At this point, I find it really hard to assess the amount of work in front of us. I can't completely nail it yet, but there is something that makes meta-circular implementation of live systems a lot harder to think about than plain layered systems with different "languages" for each of the layers. To be successful, I think we will have to learn as much as we can from other projects. Talking about tools, I think the Maxine VM inspector is really interesting. http://wikis.sun.com/display/MaxineVM/Inspector Still interesting to note that Sun lost interest in the Klein project but still kept the idea of meta-cicularity... Erick Le 29/04/10 19:30 , Marc Feeley a écrit :
Nice find Erick. Well what can I say... the reasons they mention for Klein being metacircular are pretty much what I was saying for Tachyon.
Marc
On 2010-04-29, at 5:16 PM, Erick Lavoie wrote:
"In constructing Klein, we have attempted to adhere to various principles, including object-orientation, metacircularity, heavy code reuse, reactiveness, and mirror-based reflection. We expect these principles to yield many benefits, including: • A simpler, more malleable VM (because it is written almost entirely in a high-level language). • Less replicated code (because the VM and the applications running on top of it are written in the same language and can share code, as can the VM and its development environment). • Better performance (because the VM will be able to do optimizations across the entire code base, including the VM code itself - imagine being able to inline the hotspots of the VM right up into an application). • Faster turnaround when making changes to the VM (because the VM will no longer be tied to C++ and its compilation system)." http://kleinvm.sourceforge.net/
However, the current version is not complete yet as a lot of things are yet to be done.
Erick _______________________________________________ Tachyon-list mailing list Tachyon-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/tachyon-list
There is still a couple of things that worries me. Given that they were mostly three working on Klein and that they never achieve a level of performance similar to their C++ implementation, I feel that the problem of boostrapping a system efficiently (in terms of development time) to achieve both simplicity and performance is not completely understood and therefore risky.
This worries me as well. The benefits they aim to achieve have not been demonstrated in any *objective* way, as of yet. They have spent years developing this system, and it still does not seem complete.
I can't completely nail it yet, but there is something that makes meta-circular implementation of live systems a lot harder to think about than plain layered systems with different "languages" for each of the layers.
It's the isolation of concepts that's missing. If you implement a VM in a language, to run another language, there is a clear separation between the host language and the implemented language. If the system is metacircular, things become much more blurred. Furthermore, it adds extra complexity in several areas... For example: 1. Our system, for example, will need to implement multithreading in JavaScript, if we intend for the VM itself to have a compilation thread, even though the JS spec does not intend for the language to be multithreaded. 2. We must make sure that the compiler can never break its own code while optimizing itself. We must also make sure that the GC, if written in JavaScript, cannot break itself. 3. We must carefully implement language extensions to allow for introspection and low level "unsafe" operation, even though this was never intended to be done in JavaScript. 4. We need some kind of explicit way of bootstrapping the system... We need a way for the system to generate an external image of itself.
To be successful, I think we will have to learn as much as we can from other projects. Talking about tools, I think the Maxine VM inspector is really interesting.
We need to pay extra attention to debugging. I was talking, for example, about the VM having a "debug" mode where it can generate some validation code (extra runtime checks)... But I don't know if we should launch ourself into a full-fledged GUI-based class browser, etc. We should make it possible to generate dumps of generated machine code and intermediate representations, however.
Still interesting to note that Sun lost interest in the Klein project but still kept the idea of meta-cicularity...
More or less. The Sun Java VM is implemented in C. That being said, I think they lost interest in SmallTalk and derivatives... They have their own business agenda involving Java. I think that like microkernels, meta-circularity in a VM is an idea that sounds nice in theory, but might be very hard to do well in practice... Just like no mainstream OS are truly microkernel based, no mainstream language implementations employ meta-circular VMs. I'm willing to give this a try, but I have serious doubts that we will be able to perform as well as a C based implementation, starting from a language as hard to optimize as JavaScript. I'm hoping we can get *decent* performance, and offset the compilation cost through the use of a compilation thread and compiled code caching. I think the *one* true advantage of having a meta-circular JS implementation will be the fact that some important parts of our implementation (such as dataflow analyses, code generation), will be significantly easier to write in JavaScript than in C/C++. I also know that this will be publishable if we can get it working well. - Maxime
Le 10-04-30 11:59 , chevalma@iro.umontreal.ca a écrit :
There is still a couple of things that worries me. Given that they were mostly three working on Klein and that they never achieve a level of performance similar to their C++ implementation, I feel that the problem of boostrapping a system efficiently (in terms of development time) to achieve both simplicity and performance is not completely understood and therefore risky.
This worries me as well. The benefits they aim to achieve have not been demonstrated in any *objective* way, as of yet. They have spent years developing this system, and it still does not seem complete.
I can't completely nail it yet, but there is something that makes meta-circular implementation of live systems a lot harder to think about than plain layered systems with different "languages" for each of the layers.
It's the isolation of concepts that's missing. If you implement a VM in a language, to run another language, there is a clear separation between the host language and the implemented language. If the system is metacircular, things become much more blurred. Furthermore, it adds extra complexity in several areas... For example:
Interesting, I had a teacher in Belgium who stressed the importance of separating and making explicit the meta-level from the base level. It might be the reason why McCarthy in the original LISP paper used m-expressions to talk about the meta-level and s-expressions to talk about the base-level.
1. Our system, for example, will need to implement multithreading in JavaScript, if we intend for the VM itself to have a compilation thread, even though the JS spec does not intend for the language to be multithreaded.
Do you see it as a language extension or as a library?
2. We must make sure that the compiler can never break its own code while optimizing itself. We must also make sure that the GC, if written in JavaScript, cannot break itself.
I am not clear on how that could happen. Would you give an example of it?
3. We must carefully implement language extensions to allow for introspection and low level "unsafe" operation, even though this was never intended to be done in JavaScript.
Again, how far can we go if we take a library approach? I mean, will we absolutely need to change the language semantic?
4. We need some kind of explicit way of bootstrapping the system... We need a way for the system to generate an external image of itself.
I don't think it will be that much of a problem, since other examples exist. I know the whole system in smalltalk is image-based and the Klein project succeeded in generating such an image too. Factor also use that approach, so at least there is inspiration to draw from!
To be successful, I think we will have to learn as much as we can from other projects. Talking about tools, I think the Maxine VM inspector is really interesting.
We need to pay extra attention to debugging. I was talking, for example, about the VM having a "debug" mode where it can generate some validation code (extra runtime checks)... But I don't know if we should launch ourself into a full-fledged GUI-based class browser, etc. We should make it possible to generate dumps of generated machine code and intermediate representations, however.
Still interesting to note that Sun lost interest in the Klein project but still kept the idea of meta-cicularity...
More or less. The Sun Java VM is implemented in C. That being said, I think they lost interest in SmallTalk and derivatives... They have their own business agenda involving Java.
I think that like microkernels, meta-circularity in a VM is an idea that sounds nice in theory, but might be very hard to do well in practice... Just like no mainstream OS are truly microkernel based, no mainstream language implementations employ meta-circular VMs.
According to the wikipedia article, performance is indeed still an issue for micro-kernels, but reliability and security have been achieved, so success actually depends on the criteria we use to measure. Still according to wikipedia, QNX is a commercial microkernel-OS that has been used in automotive systems and RIM have announced recently that they plan to acquire it, so it might also find its way into cell phones. As an afterthought, I am not sure that we should judge the quality of an idea on its mainstream acceptance at any given point in time. I guess Mark will agree on this one ( scheme... wink wink).
I'm willing to give this a try, but I have serious doubts that we will be able to perform as well as a C based implementation, starting from a language as hard to optimize as JavaScript. I'm hoping we can get *decent* performance, and offset the compilation cost through the use of a compilation thread and compiled code caching.
I think the *one* true advantage of having a meta-circular JS implementation will be the fact that some important parts of our implementation (such as dataflow analyses, code generation), will be significantly easier to write in JavaScript than in C/C++. I also know that this will be publishable if we can get it working well.
While still being worried, I do think that the fact that we do not understand how to do it well is an indicator of a worthwhile line of research instead of an exercise in engineering.
- Maxime _______________________________________________ Tachyon-list mailing list Tachyon-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/tachyon-list
participants (2)
-
chevalma＠iro.umontreal.ca -
Erick Lavoie