Xcode 6.1.1 was released a little over a month ago, so I decided to do some benchmarking to see how well its C compilers (Apple clang 6.0 and Apple gcc 6.0) compare against GNU gcc 4.9.2 when compiling Gambit.
I configured and built Gambit with
./configure --enable-single-host CC="<C_COMPILER>" make -j8
I report the build time for each compiler and the execution time of test 4 (which measures the speed of the interpreter, which is indicative of the relative speed of the code generated by Gambit). Here are the numbers:
Build test4 C compiler (secs) (secs)
60.31 1.30 GNU gcc 4.9.2 2042.98 (34x) 3.05 (2.3x) Apple clang 6.0 1875.56 (31x) 3.08 (2.4x) Apple gcc 6.0
As can be seen, the Apple C compilers build Gambit in roughly 30 minutes, which is about 30x slower than when using GNU gcc 4.9.2 . The execution time is also better for GNU gcc, about 2.3x faster.
So on OS X it is clearly advantageous to install GNU gcc when using Gambit. This can easily be achieved with the homebrew system with the command
brew install gcc49
Marc
P.S. here is the version information of the compilers:
============================================================================== GNU gcc 4.9.2:
% /usr/local/Cellar/gcc49/4.9.2/bin/gcc-4.9 -v Using built-in specs. COLLECT_GCC=/usr/local/Cellar/gcc49/4.9.2/bin/gcc-4.9 COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc49/4.9.2/libexec/gcc/x86_64-apple-darwin14.0.0/4.9.2/lto-wrapper Target: x86_64-apple-darwin14.0.0 Configured with: ../configure --build=x86_64-apple-darwin14.0.0 --prefix=/usr/local/Cellar/gcc49/4.9.2 --enable-languages=c,c++,objc,obj-c++ --program-suffix=-4.9 --with-gmp=/usr/local/opt/gmp4 --with-mpfr=/usr/local/opt/mpfr2 --with-mpc=/usr/local/opt/libmpc08 --with-cloog=/usr/local/opt/cloog018 --with-isl=/usr/local/opt/isl011 --with-system-zlib --enable-version-specific-runtime-libs --enable-libstdcxx-time=yes --enable-stage1-checking --enable-checking=release --enable-lto --disable-werror --with-pkgversion='Homebrew gcc49 4.9.2' --with-bugurl=https://github.com/Homebrew/homebrew-versions/issues --enable-plugin --disable-nls --enable-multilib Thread model: posix gcc version 4.9.2 (Homebrew gcc49 4.9.2)
============================================================================== Apple clang 6.0:
% /usr/bin/clang -v Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn) Target: x86_64-apple-darwin14.0.0 Thread model: posix
============================================================================== Apple gcc 6.0:
% /usr/bin/gcc -v Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1 Apple LLVM version 6.0 (clang-600.0.56) (based on LLVM 3.5svn) Target: x86_64-apple-darwin14.0.0 Thread model: posix
==============================================================================
Afficher les réponses par date
On Wed, Jan 14, 2015 at 8:21 PM, Marc Feeley feeley@iro.umontreal.ca wrote:
Xcode 6.1.1 was released a little over a month ago, so I decided to do some benchmarking to see how well its C compilers (Apple clang 6.0 and Apple gcc 6.0) compare against GNU gcc 4.9.2 when compiling Gambit.
I configured and built Gambit with
./configure --enable-single-host CC="<C_COMPILER>" make -j8
I report the build time for each compiler and the execution time of test 4 (which measures the speed of the interpreter, which is indicative of the relative speed of the code generated by Gambit). Here are the numbers:
Build test4 C compiler (secs) (secs)
60.31 1.30 GNU gcc 4.9.2
2042.98 (34x) 3.05 (2.3x) Apple clang 6.0 1875.56 (31x) 3.08 (2.4x) Apple gcc 6.0
Thank you for the benchmarks, Marc. It seems that GCC still greatly outperforms Apple's GCC/Clang with Gambit code. Do you have any idea if it is a particular optimization that is producing this divergence, or is just the accumulation of small differences?
On Jan 16, 2015, at 5:15 AM, Álvaro Castro-Castilla alvaro.castro.castilla@gmail.com wrote:
On Wed, Jan 14, 2015 at 8:21 PM, Marc Feeley feeley@iro.umontreal.ca wrote: Xcode 6.1.1 was released a little over a month ago, so I decided to do some benchmarking to see how well its C compilers (Apple clang 6.0 and Apple gcc 6.0) compare against GNU gcc 4.9.2 when compiling Gambit.
I configured and built Gambit with
./configure --enable-single-host CC="<C_COMPILER>" make -j8
I report the build time for each compiler and the execution time of test 4 (which measures the speed of the interpreter, which is indicative of the relative speed of the code generated by Gambit). Here are the numbers:
Build test4 C compiler (secs) (secs)
60.31 1.30 GNU gcc 4.9.2
2042.98 (34x) 3.05 (2.3x) Apple clang 6.0 1875.56 (31x) 3.08 (2.4x) Apple gcc 6.0
Thank you for the benchmarks, Marc. It seems that GCC still greatly outperforms Apple's GCC/Clang with Gambit code. Do you have any idea if it is a particular optimization that is producing this divergence, or is just the accumulation of small differences?
My guess is that clang is using optimizations of high computational complexity, and the development team does not do performance testing on programs with very large C functions. I remember that some old versions of GNU gcc (version 3.2?) also had slow compile times for Gambit, but not as dramatic as with clang which is 30x slower. Fortunately, Brad Lucier was in contact with the GNU gcc development team and put in the effort to pinpoint the origin of the problem and eventually fix the issue. Someone with contacts on the LLVM development team should do that for clang!
Marc