[gambit-list] More benchmarks

Marc Feeley feeley at iro.umontreal.ca
Thu Oct 19 13:36:15 EDT 2006


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


On 19-Oct-06, at 12:44 PM, MichaelL at frogware.com wrote:

>
> Hello, Marc & Will.
>
> For what it's worth, when I last looked at the way Chez & Petite  
> were run I noticed that it wasn't quite correct. The script I saw  
> does this:
>
>         chez_exec ()
>         {
>           echo "(load \"$1.scm\")" | /usr/bin/time scheme
>         }
>
>         petite_chez_exec ()
>         {
>           echo "(load \"$1.scm\")" | /usr/bin/time petite
>         }
>
> More correct would be something like this:
>
>         "(optimize-level 2) (module () (import-only scheme)  
> (include \"$1.scm\"))"
>
> The "(optimize-level 2)" will enable immutable Scheme bindings (and  
> primitive inlining) while still producing safe code. The top-level  
> module (especially with the absence of any exports) will signal to  
> Chez that it is free to inline. The "(import-only scheme)"  
> declaration is virtually identical to the "(optimize-level 2)"  
> declaration, at least in Chez 7, and is in fact preferred over  
> setting the optimize level--but Will is using 6.1 so I'd recommend  
> using both since it will do no harm and 6.1 might be different.  
> (See http://www.scheme.com/csug7/use.html#./use:h3 for more  
> information on all of this.) Btw, a quick look at the Scheme code  
> shows this change will create at least one problem because  
> expressions sometimes appear before definitions (eg, the "(setup)"  
> in boyer.scm before the definition of "main.").
>
> Finally, make sure you aren't measuring compilation time in  
> addition to running time. The above approach appears to be doing  
> just that; for Chez at least compilation time can be significant  
> relative to a small running time. Optimizations take time. :-)
>
> If you're interested in more Chez times we have Chez 7.1 for  
> Windows, Linux, Solaris, and Mac. (Our Sun box is rather ancient  
> though.)
>
> (Apologies if I misread something about the way the tests were set  
> up.)

I do not have access to Chez Scheme.  However Will does, and he has  
modified the "bench" script to call up Chez Scheme this way:

   r6rs)
     INSERTCODE="(define-syntax if-fixflo (syntax-rules () ((if- 
fixflo yes no) no)))"
     REPLCOMMANDS="(begin (optimize-level 2) (load \"%s.scm\") (main))"
     ;;

So the (optimize-level 2) declaration is being used, but not the  
(module () ...) wrapper.  It is possible that Will could not use that  
trick because some benchmarks could no longer be compiled because  
expressions appear before definitions as you mention.  I'll let Will  
look into this since I don't have access to Chez Scheme.

Concerning timing, the Chez Scheme prefix file uses the "(time  
<expr>)" form to measure the execution time, so the compile time is  
not included.  In the new "bench" script /usr/bin/time is not used  
for Chez Scheme.

If you can run the benchmarks, I would be interested in seeing how  
Gambit compares to Chez Scheme 7.1 on Linux and Macintel.  I can send  
you the latest benchmark suite.

Marc


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.3 (Darwin)

iD8DBQFFN7eP//V9Zc2T/v4RApexAJ91PLWsLpyFaucZLq8IzE66aP3QTQCgoWj7
0RPxKCtScKOhDnt9j9RZMZ4=
=lXns
-----END PGP SIGNATURE-----



More information about the Gambit-list mailing list