On Wed, 2010-03-03 at 12:32 -0500, David St-Hilaire wrote:
Bradley Lucier wrote:
It is, but I can't get benchmark.scm to run, even with a bit of hacking. Missing directories, files, getting errors in symbol-append, things like that.
This file is very old... I don't even remember writing it hehe. Please don't mind the benchmark.scm file in the class repository. If your interested about the benchmarks, they are presented in the thesis and the source code of thoses benchmarks can be found in my thesis repo: http://github.com/sthilaid/memoire/tree/master/redaction/benchmarks/
Indeed, but bench.scm has dependencies that aren't in the class git directory (or anywhere else I can find). I can't get it to run.
I redid your define-type and meroon benchmarks to (a) put the definitions in separate files that are compiled separately, and (b) use the same bench.scm file for both. I added declarations to the file bench.scm because I want fx+ and fl- to be inlined. I didn't add any declarations to the files that defined the "class"es. I also set a minimum heap of 100,000KB because I didn't want to be benchmarking Gambit's garbage collector.
I wanted to measure the speed when the "class" definitions are in a different file from the file that uses them. Maybe you wanted to measure something different.
All the files are below.
The results are:
Define-type:
heine:~/Desktop/oop-bench> gsi -:m100000 Gambit v4.6.0
(load "define-type-definitions")
"/home/lucier/Desktop/oop-bench/define-type-definitions.o2"
(load "bench")
(instance-creation .1758890151977539) (access .06786298751831055) (modif .06581807136535645) (dispatch-2 .1283562183380127) (dispatch-5 .29570913314819336) (polymorhpic-dispatch-2 .13241195678710938) (polymorhpic-dispatch-5 .29534316062927246) "/home/lucier/Desktop/oop-bench/bench.o2"
Meroon:
heine:~/Desktop/oop-bench> gsi++ -:m100000 [ Meroon V3 Paques2001+1 $Revision: 1.2 $ ] Gambit v4.6.0
(load "meroon-definitions")
"/home/lucier/Desktop/oop-bench/meroon-definitions.o2"
(load "bench")
(instance-creation .10095787048339844) (access .07755494117736816) (modif .07986021041870117) (dispatch-2 .26418304443359375) (dispatch-5 .31914305686950684) (polymorhpic-dispatch-2 .2722489833831787) (polymorhpic-dispatch-5 .39966797828674316) "/home/lucier/Desktop/oop-bench/bench.o2"
So I measure the ratio between using define-type and meroon (>1 means define-type is faster, < 1 means meroon is faster):
(instance-creation .5739862172170924) (access 1.142816489716763) (modif 1.2133477745860517) (dispatch-2 2.0582021490995888) (dispatch-5 1.0792465334831902) (polymorhpic-dispatch-2 2.0560755236092305) (polymorhpic-dispatch-5 1.3532325496726967)
So I don't see much of a speed benefit to use define-type as compared to Meroon; this is quite different from the results in your thesis, so I suspect that you measured something quite different in your thesis.
Brad
Afficher les réponses par date
I've wondered about the results in the thesis, too; note that unsafe compilation matters very much for Gambit's records. The type checks are very expensive (maybe more expensive than 'necessary'). So I've been thinking that David maybe ran some tests in unsafe mode and not others.
Christian.
They were indeed. ;)
-- David
Le 2010-03-05 à 15:03, Christian Jaeger chrjae@gmail.com a écrit :
I've wondered about the results in the thesis, too; note that unsafe compilation matters very much for Gambit's records. The type checks are very expensive (maybe more expensive than 'necessary'). So I've been thinking that David maybe ran some tests in unsafe mode and not others.
Christian.
On Fri, 2010-03-05 at 14:03 -0500, Bradley Lucier wrote:
Indeed, but bench.scm has dependencies that aren't in the class git directory (or anywhere else I can find). I can't get it to run.
Sorry, I mean class.scm includes scm-lib.scm, which I could not find.
Brad
On Mar 5, 2010, at 3:56 PM, Bradley Lucier wrote:
Sorry, I mean class.scm includes scm-lib.scm, which I could not find.
Brad
http://github.com/sthilaid/scm-lib
I had similar problems so I went to the root of his github source tree and downloaded everything.
warmest regards, Ralph
Raffael Cavallaro raffaelcavallaro@me.com
Sorry for the trouble...! I guess that my modular build system is pretty amateur work x_x'.
-- David
Le 2010-03-05 à 16:15, Raffael Cavallaro raffaelcavallaro@mac.com a écrit :
On Mar 5, 2010, at 3:56 PM, Bradley Lucier wrote:
Sorry, I mean class.scm includes scm-lib.scm, which I could not find.
Brad
http://github.com/sthilaid/scm-lib
I had similar problems so I went to the root of his github source tree and downloaded everything.
warmest regards, Ralph
Raffael Cavallaro raffaelcavallaro@me.com
Gambit-list mailing list Gambit-list@iro.umontreal.ca https://webmail.iro.umontreal.ca/mailman/listinfo/gambit-list
On Mar 5, 2010, at 4:32 PM, David St-Hilaire wrote:
Sorry for the trouble...! I guess that my modular build system is pretty amateur work x_x'.
No trouble at all - thanks for sharing all of this with us.
warmest regards,
Ralph
Raffael Cavallaro raffaelcavallaro@me.com
Ah, OK, thank you all. Now I get results similar to David for (a) including definitions and benchmark code in the same file, and (b) using the declarations:
(declare (standard-bindings) (extended-bindings) (block) (not safe))
frying-pan:~/Desktop/oop-bench> gsi -:m100000 class-benchmark (class-direct-instance-creation .003332853317260742) (class-constructor-instance-creation .9427149295806885) (class-access .006258964538574219) (class-modif .00386810302734375) (class-dispatch-2 .7867801189422607) (class-dispatch-5 1.447295904159546) (class-polymorhpic-dispatch-2 2.3654139041900635) (class-polymorhpic-dispatch-5 5.206311225891113)
frying-pan:~/Desktop/oop-bench> gsi -:m100000 define-type-benchmark (define-type-instance-creation .003023862838745117) (define-type-access .004261016845703125) (define-type-modif .004247903823852539) (define-type-dispatch-2 .06431698799133301) (define-type-dispatch-5 .21142292022705078) (define-type-polymorhpic-dispatch-2 .10065913200378418) (define-type-polymorhpic-dispatch-5 .2658059597015381)
frying-pan:~/Desktop/oop-bench> gsi++ -:m100000 meroon-benchmark [ Meroon V3 Paques2001+1 $Revision: 1.1 $ ] (meroon-instance-creation .07878613471984863) (meroon-constructor-instance-creation .07859683036804199) (meroon-access .05495786666870117) (meroon-modif .048612117767333984) (meroon-dispatch-2 .24296903610229492) (meroon-dispatch-5 .27795886993408203) (meroon-polymorhpic-dispatch-2 .25885701179504395) (meroon-polymorhpic-dispatch-5 .3787720203399658)
I was trying to figure out what "new" in class.scm does different from just make-*; it appears to call a constructor (if one is available) for certain fields. I think that's what the "instantiate" macro is intended to do in Meroon; if an explicit value is not given for a slot, then it calls a constructor. So I included a benchmark for making an instance using the "instantiate" macro (which doesn't make any difference for these simple classes).
I think I understand why Meroon is noticeably slower for a number of operations:
1. As part of each instance creation, Meroon calls the initialize! generic on the newly created instance. That's a cross-trampoline call to a generic, in this case for a no-op.
2. Each access and modification is "safe", in that it checks whether its argument is of the same class (or of a subclass). This is also a cross-trampoline call to a generic. I suppose it might be better if the Gambit compiler exposed the declarations to the macro-expander, so one could change the macro expansion of define-class to choose whether to insert the checks based on the current declaration of (safe) or (not safe).
3. The Meroon runtime is not in the same file, so that affects dispatch.
Thanks for the interesting benchmarks.
Brad
This thread got me wondering whether "unsafe" Meroon accessors, setters, etc., would be of interest to people.
On the machine I'm sitting in front of, I get with the current Meroon
(meroon-instance-creation .15108799934387207) (meroon-access .041027069091796875) (meroon-modif .04129195213317871) (meroon-dispatch-2 .5176689624786377) (meroon-dispatch-5 .9149508476257324) (meroon-polymorhpic-dispatch-2 .5540168285369873) (meroon-polymorhpic-dispatch-5 1.0130860805511475)
If I define a new file _meroon#.scm that contains everything needed for check-class and include it in the benchmark file, I get
(meroon-instance-creation .16762185096740723) (meroon-access .012614011764526367) (meroon-modif .011465072631835938) (meroon-dispatch-2 .48472118377685547) (meroon-dispatch-5 1.0633580684661865) (meroon-polymorhpic-dispatch-2 .4715709686279297) (meroon-polymorhpic-dispatch-5 1.0170669555664062)
So, it's a bit over three times as fast.
But if I rewrite the getter/setter/... code to allow the user to define a macro
(define-macro (meroon-safe-fields?) #f)
that eliminates the runtime checks I get
(meroon-instance-creation .16996002197265625) (meroon-access .002710103988647461) (meroon-modif .0028591156005859375) (meroon-dispatch-2 .5442471504211426) (meroon-dispatch-5 .9221928119659424) (meroon-polymorhpic-dispatch-2 .49954700469970703) (meroon-polymorhpic-dispatch-5 .9451491832733154)
and with define-type I get
(define-type-instance-creation .0019330978393554688) (define-type-access .003993988037109375) (define-type-modif .002807140350341797) (define-type-dispatch-2 .047636985778808594) (define-type-dispatch-5 .13761401176452637) (define-type-polymorhpic-dispatch-2 .06707501411437988) (define-type-polymorhpic-dispatch-5 .17751502990722656)
So accessing and modifying is about as fast as with define-type; instance-creation is more, but that's because instance creation calls the intialize! generic on the created instance, and I don't want to give that up. (After all, Meroon should offer *something* beyond define-type structures.)
So, what do you think? The macro definition
(define-macro (meroon-safe-fields?) #f)
would apply to the current file, meaning something like
(declare (not meroon-safe-fields))
Is it so important to have fast unsafe access and modification functions?
Brad
Hallo,
On Fri, Mar 12, 2010 at 9:38 PM, Bradley Lucier lucier@math.purdue.edu wrote:
So, what do you think? The macro definition
(define-macro (meroon-safe-fields?) #f)
would apply to the current file, meaning something like
(declare (not meroon-safe-fields))
Is it so important to have fast unsafe access and modification functions?
I am currently not using Meroon but I plan to, possibly in 3D computer graphics. Certainly for a 3D scene graph fast access is important and, after all, it's customisable.
Cheers,
Wow, this sure is a great performance gain! It would be conveniant that a (not safe) declaration would automatically do this, but still this is a pretty convincing benchmark to use Meroon (as long as multiple hierarchy is not desired ^_-).
-- David
Le 2010-03-12 à 16:38, Bradley Lucier lucier@math.purdue.edu a écrit :
This thread got me wondering whether "unsafe" Meroon accessors, setters, etc., would be of interest to people.
On the machine I'm sitting in front of, I get with the current Meroon
(meroon-instance-creation .15108799934387207) (meroon-access .041027069091796875) (meroon-modif .04129195213317871) (meroon-dispatch-2 .5176689624786377) (meroon-dispatch-5 .9149508476257324) (meroon-polymorhpic-dispatch-2 .5540168285369873) (meroon-polymorhpic-dispatch-5 1.0130860805511475)
If I define a new file _meroon#.scm that contains everything needed for check-class and include it in the benchmark file, I get
(meroon-instance-creation .16762185096740723) (meroon-access .012614011764526367) (meroon-modif .011465072631835938) (meroon-dispatch-2 .48472118377685547) (meroon-dispatch-5 1.0633580684661865) (meroon-polymorhpic-dispatch-2 .4715709686279297) (meroon-polymorhpic-dispatch-5 1.0170669555664062)
So, it's a bit over three times as fast.
But if I rewrite the getter/setter/... code to allow the user to define a macro
(define-macro (meroon-safe-fields?) #f)
that eliminates the runtime checks I get
(meroon-instance-creation .16996002197265625) (meroon-access .002710103988647461) (meroon-modif .0028591156005859375) (meroon-dispatch-2 .5442471504211426) (meroon-dispatch-5 .9221928119659424) (meroon-polymorhpic-dispatch-2 .49954700469970703) (meroon-polymorhpic-dispatch-5 .9451491832733154)
and with define-type I get
(define-type-instance-creation .0019330978393554688) (define-type-access .003993988037109375) (define-type-modif .002807140350341797) (define-type-dispatch-2 .047636985778808594) (define-type-dispatch-5 .13761401176452637) (define-type-polymorhpic-dispatch-2 .06707501411437988) (define-type-polymorhpic-dispatch-5 .17751502990722656)
So accessing and modifying is about as fast as with define-type; instance-creation is more, but that's because instance creation calls the intialize! generic on the created instance, and I don't want to give that up. (After all, Meroon should offer *something* beyond define- type structures.)
So, what do you think? The macro definition
(define-macro (meroon-safe-fields?) #f)
would apply to the current file, meaning something like
(declare (not meroon-safe-fields))
Is it so important to have fast unsafe access and modification functions?
Brad
I agree, this is definitely an option that Meroon should provide!
On Fri, Mar 12, 2010 at 4:51 PM, David St-Hilaire sthilaid@iro.umontreal.ca wrote:
Wow, this sure is a great performance gain!
-
Alex Queiroz -
Bradley Lucier -
Christian Jaeger -
David St-Hilaire -
James Long -
Raffael Cavallaro