Gambit has reached a milestone today with the implementation of the parallel garbage collector. This is an important step towards truly concurrent Gambit threads.
When Gambit is configured with --enable-multiple-threaded-vms the available processors will cooperate to do the garbage collection in parallel. It is still a “stop the world” GC, but the operation of the GC uses the available parallelism. By default the runtime system will use N processors where N is one less than the number of physical CPUs, but the -:pN runtime option can be used to specify the number of processors. This can be useful for GC intensive programs, such as the Gambit compiler itself. Here is an example with 1 and 4 processors to show how the GC time is affected:
% time gsc -:d2,p1 -c compiler.scm *** GC: 2 ms, 2.91M alloc, 6.02M heap, 1015K live (16% 1019136+20368) *** GC: 2 ms, 4.97M alloc, 7.02M heap, 1.95M live (28% 2023824+22352) *** GC: 2 ms, 7.03M alloc, 10.0M heap, 3.18M live (32% 3312304+22352) *** GC: 4 ms, 10.7M alloc, 15.0M heap, 5.33M live (35% 5569040+22352) *** GC: 7 ms, 16.8M alloc, 23.0M heap, 8.99M live (39% 9405296+22352) *** GC: 19 ms, 26.7M alloc, 42.0M heap, 17.1M live (41% 17870848+20184) *** GC: 34 ms, 46.3M alloc, 74.0M heap, 31.0M live (42% 32506560+20184) *** GC: 62 ms, 81.7M alloc, 124M heap, 53.1M live (43% 55680400+20184) *** GC: 127 ms, 142M alloc, 212M heap, 91.4M live (43% 95797840+20184) *** GC: 141 ms, 245M alloc, 270M heap, 117M live (43% 122309392+41688) *** GC: 88 ms, 376M alloc, 271M heap, 117M live (43% 123146832+41688) *** GC: 151 ms, 508M alloc, 326M heap, 141M live (43% 147992016+20184) *** GC: 204 ms, 668M alloc, 397M heap, 172M live (43% 180593424+20184) *** GC: 140 ms, 854M alloc, 307M heap, 133M live (43% 139397056+68968) *** GC: 178 ms, 1003M alloc, 401M heap, 174M live (43% 182417376+138192) *** GC: 216 ms, 1.17G alloc, 523M heap, 228M live (43% 238330912+226440) *** GC: 290 ms, 1.42G alloc, 683M heap, 297M live (44% 310766080+1166104) *** GC: 251 ms, 1.74G alloc, 775M heap, 338M live (44% 351564560+2824440) *** GC: 246 ms, 2.11G alloc, 775M heap, 338M live (44% 351564544+2824440)
real 0m7.137s user 0m6.652s sys 0m0.450s % time gsc -:d2,p4 -c compiler.scm *** GC: 0 ms, 371K alloc, 12.0M heap, 50.9K live (0% 50864+1288) *** GC: 1 ms, 3.24M alloc, 12.0M heap, 1.02M live (8% 1049488+20536) *** GC: 1 ms, 5.75M alloc, 14.0M heap, 2.42M live (17% 2515584+22520) *** GC: 1 ms, 9.56M alloc, 20.0M heap, 4.68M live (23% 4880400+22520) *** GC: 2 ms, 15.4M alloc, 28.0M heap, 8.18M live (29% 8551792+22520) *** GC: 5 ms, 25.8M alloc, 46.0M heap, 16.3M live (35% 17093312+20352) *** GC: 9 ms, 45.3M alloc, 78.0M heap, 30.4M live (39% 31834640+20352) *** GC: 16 ms, 80.6M alloc, 129M heap, 52.5M live (41% 54979856+20352) *** GC: 26 ms, 141M alloc, 217M heap, 91.1M live (42% 95544320+20352) *** GC: 34 ms, 245M alloc, 276M heap, 117M live (42% 122290592+41856) *** GC: 26 ms, 378M alloc, 278M heap, 118M live (42% 123178512+41856) *** GC: 37 ms, 512M alloc, 335M heap, 142M live (43% 149378784+20352) *** GC: 45 ms, 675M alloc, 410M heap, 176M live (43% 184114144+20352) *** GC: 39 ms, 865M alloc, 320M heap, 136M live (43% 142687856+72960) *** GC: 42 ms, 1018M alloc, 418M heap, 179M live (43% 187690608+142128) *** GC: 60 ms, 1.19G alloc, 539M heap, 232M live (43% 243163808+226608) *** GC: 73 ms, 1.44G alloc, 710M heap, 307M live (43% 320299328+1546624) *** GC: 74 ms, 1.78G alloc, 781M heap, 338M live (43% 351571728+2824608) *** GC: 68 ms, 2.15G alloc, 781M heap, 338M live (43% 351572144+2824608)
real 0m5.515s user 0m6.682s sys 0m0.494s
This shows that for large heaps the GC speeds up almost linearly with the number of processors.
The parallel GC is fairly well debugged, but I don't recommend it for production use until it has been tested by more users in a variety of situations and operating systems.
So at this point I am calling upon users to experiment with using the parallel GC and to report back any problems. Just configure Gambit with:
% ./configure --enable-multiple-threaded-vms
You can of course add --enable-single-host for more speed and other configure options. It is also interesting to add --enable-activity-log so that an activity log of the parallel execution (gambit.actlog) is generated. This file can be viewed with the xactlog program that has to be compiled separately:
% make xactlog % contrib/xactlog/xactlog gambit.actlog
Below is the execution profile generated by xactlog for the previous example with 4 processors. The green bars indicate the execution of the main program, and the red bars indicate the execution of the GC which can be seen to execute on all 4 processors (top part of the profile).
Marc
