Hi Brad.

You wrote, 2017-11-28 13:41: [...]

I started a draft of a note about scheme benchmarking, in the specific context of the Gambit benchmarks, Larceny's R6RS benchmarks, and the R7RS benchmarks on ecraven's site.

I'm not terribly happy with it as it stands, so I don't know how much more I'll develop it, but I'm soliciting comments on the current draft.

Many Scheme implementations perform differently for different ABIs: x86-64 is often _slower_ than x32 ( https://en.wikipedia.org/wiki/X32_ABI ). Here are fresh numbers (bigloo 4.3b (alpha), on r7rs-benchmarks, https://github.com/ecraven/r7rs-benchmarks ; similar findings for Chicken):

(The final real number in each of the two columns is the runtime in seconds.)

x86-64 x32 browse:2000,5.19260288 2000,2.880740096 deriv:10000000,6.965362944 10000000,3.716836864 destruc:600:50:4000,5.549836032 600:50:4000,3.574226176 diviter:1000:1000000,6.392005888 1000:1000000,3.6317312 divrec:1000:1000000,10.001812992 1000:1000000,6.718710016 puzzle:1000,13.80742784 1000,8.90971008 triangl:22:1:50,4.011372032 22:1:50,2.866685952 tak:40:20:11:1,4.730241792 40:20:11:1,3.425227008 takl:40:20:12:1,2.600088064 40:20:12:1,2.011789056 ntakl:40:20:12:1,2.400090112 40:20:12:1,1.674188032 cpstak:40:20:11:1,24.854480896 40:20:11:1,12.031755008 fib:40:5,7.065795072 40:5,6.519035136 fibc:30:10,90.011451136 30:10,74.887549952 fibfp:35.0:10,5.812903936 35.0:10,6.09537408 sum:10000:200000,9.243782912 10000:200000,8.300467968 sumfp:1000000.0:500,11.34256896 1000000.0:500,12.28169088 fft:65536:100,4.25339392 65536:100,4.51265408 mbrot:75:1000,15.3834688 75:1000,15.12131712 nucleic:50,4.15466496 50,4.18435584 pnpoly:1000000,12.179164928 1000000,12.505131008 ray:50,10.29706496 50,8.56330112 simplex:1000000,4.91090176 1000000,5.237882112 ack:3:12:2,5.353505024 3:12:2,5.52949504 array1:1000000:500,4.774448896 1000000:500,4.565778944 string:500000:25,0.112798976 500000:25,0.135549952 sum1:25,0.554072064 25,0.521470976 cat:50,11.41799424 50,7.51307008 tail:25,4.632050944 25,3.492803072 read1:2500,1.66528896 read1:2500,1.621330944 conform:500,2.121995008 500,1.891009024 dynamic:500,3.746084864 500,3.786791168 earley:1,5.485622016 1,4.296999936 graphs:7:3,18.115545088 7:3,15.547826944 matrix:5:5:2500,3.149252096 5:5:2500,2.635922944 maze:20:7:10000,2.244920064 20:7:10000,1.58433792 mazefun:11:11:10000,4.158912 11:11:10000,3.929737984 nqueens:13:10,14.757040128 13:10,13.200544 paraffins:23:10,2.04002816 23:10,1.92710784 parsing:2500,2.118763008 2500,2.19452416 peval:2000,1.818642176 2000,1.665608192 primes:1000:10000,3.791961088 1000:10000,2.637708032 quicksort:10000:2500,4.817225216 10000:2500,4.884603136 scheme:100000,3.935532032 100000,3.266596096 slatex:500,3782.291475968 500,2936.8582912 nboyer:5:1,3.554392832 5:1,2.905526016 sboyer:5:1,1.169430016 5:1,1.055747072 gcbench:20:1,2.098480896 20:1,2.074130176 mperm:20:10:2:1,7.560857856 20:10:2:1,7.810764032 bv2string:1000:1000:100,3.837786112 1000:1000:100,2.59586816

So, ABI might become an interesting aspect of your evaluation.

Ciao Sven