[gambit-list] Gambit-C vs. Bigloo vs. MzScheme

Marc Feeley feeley at iro.umontreal.ca
Sun Aug 27 18:38:30 EDT 2006


Based on some of the feedback I have updated the Gambit benchmark  
suite (http://www.iro.umontreal.ca/~gambit/bench.tar.gz) and the  
result page (http://www.iro.umontreal.ca/~gambit/bench.html).  This  
time the benchmarks were run in three different situations.  This was  
done by using compiler options and/or declarations.

- Safe with generic arithmetic
     - generic arithmetic operations
     - overflow detection on fixnum arithmetic (either produce
       a bignum or signal an exception)
     - immutable bindings for the definitions in the benchmark
     - immutable predefined bindings (for +, car, ...)
     - safe execution (i.e. an exception must be signalled on errors)

- Safe with fixnum/flonum specializations
     - arithmetic operations are specialized to fixnum or flonum
       arguments as appropriate (for example turning + into fl+)
       and the fixnum operations may wrap on overflow
     - immutable bindings for the definitions in the benchmark
     - immutable predefined bindings (for +, car, ...)
     - safe execution (i.e. an exception must be signalled on errors)

- Unsafe with fixnum/flonum specializations
     - like ``Safe with fixnum/flonum specializations'' but errors are
       not checked

I only ran MzScheme in the ``Safe with generic arithmetic'' situation  
because I didn't have time to figure out what was needed to get it to  
work in the other situations (actually I don't think MzScheme has an  
unsafe mode, but I could be wrong).

Anyway, in half of the benchmarks Gambit-C is faster than Bigloo when  
using fixnum/flonum specializations (in the safe and unsafe  
situations).  For some benchmarks which use call/cc Gambit-C is 50 to  
60 times faster than Bigloo.  On floating point intensive code,  
Gambit-C can be 12 times slower than Bigloo but sometimes 3 times  
faster.  The worst case for Gambit-C is the "string" benchmark which  
is 15 times slower (a large part of the slowdown is due to the string  
representation, Gambit-C uses 4 bytes per char and Bigloo 1 byte per  
char; this could be improved by recompiling Gambit-C so that strings  
use 1 or 2 bytes per char).

Of course, benchmarks are benchmarks.  These measurements may change  
(sometimes dramatically) between different versions of the systems.   
So please take this with a big grain of salt.

Marc




More information about the Gambit-list mailing list