[gambit-list] Gambit-C vs. Bigloo vs. MzScheme

Marc Feeley feeley at iro.umontreal.ca
Sat Aug 26 21:28:15 EDT 2006


After I saw Eli Barzilay's post to comp.lang.scheme about the  
performance of MzScheme compared to other Scheme systems (http:// 
groups.google.com/group/comp.lang.scheme/msg/473f91e7065b6f73? 
dmode=source) I was tempted to do some benchmarking of my own using  
the Gambit Scheme benchmark suite which goes beyond the classical  
Gabriel benchmarks (it has close to 60 Scheme benchmarks).  Most  
benchmarks are straightforward R4RS code and so should work on just  
about any Scheme system.

I have updated the benchmarking infrastructure to accomodate other  
Scheme systems, and MzScheme and Bigloo in particular.  I also wrote  
a script to run all benchmarks and automatically generate an HTML  
table of the results.  The table can be viewed here:

    http://www.iro.umontreal.ca/~gambit/bench.html

And the benchmark suite is available here:

    http://www.iro.umontreal.ca/~gambit/bench.tar.gz

As you can see from the table, Gambit-C is faster than Bigloo and  
MzScheme on roughly 3 out of 4 benchmarks.  I have used the same  
assumptions as Eli, which approximate the semantics of R6RS (see the  
last section of the result page for details).

It would be really interesting to extend this experiment to other  
Scheme systems.  It should not be too difficult to modify the  
benchmarking scripts to test other systems.  The hardest parts are  
installing the Scheme system, figuring out which compiler options are  
equivalent to the ones used by the other systems (i.e. approximate  
R6RS semantics), and work out unexpected problems.

The following systems are particularly interesting to compare to:

   - Chicken
   - Scheme 48
   - Larceny
   - Chez Scheme

Are there any volunteers?

Marc




More information about the Gambit-list mailing list