[gambit-list] Real-time or generational gc?

sarna sarna.dev at protonmail.com
Sat Nov 9 12:26:04 EST 2019


Hey all,

I'm a Scheme noob, please be gentle :)

Yes, the Racket code isn't equivalent - it uses immutable hashes instead of mutable ones. I couldn't find immutable ones in Gerbil's docs. Mutable ones should be less of a concern for the GC though, right?

The code was adopted from this blog post:
http://prl.ccs.neu.edu/blog/2016/05/24/measuring-gc-latencies-in-haskell-ocaml-racket/

The Gerbil code itself:

(import :std/iter)
(import :gerbil/gambit/os)
;; (declare (fixnum))
(gc-report-set! #t)
;; (export main)

(define window-size 200000)
(define msg-count 2000000)

(define (message n) (make-u8vector 1024 (modulo n 256)))

(define (push-msg chan id-high)
  (define id-low (- id-high window-size))
  (define inserted
    (begin
      (hash-put! chan id-high (message id-high))
      chan))
  (if (hash-get inserted (< id-low 0))
      (hash-remove! inserted id-low)))

(define (main)
  (time
    (for/fold
    (chan (make-hash-table))
    (i (in-range msg-count))
    (begin
      (push-msg chan i)
      chan))))

(main)

Running it with gxi produces a peak ~840ms (!) pause. Compiling it with gxc (after commenting (main) and uncommenting (export main)) and running it produces a peak 637ms pause - that's still more than half a second.

-- sarna

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, November 9, 2019 4:50 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:

> CC'ing sarna, who found the pathology.
> The u8vectors were smaller than the still limit, so they were being moved around.
>
> -- vyzo
>
> On Sat, Nov 9, 2019 at 5:44 PM Marc Feeley <feeley at iro.umontreal.ca> wrote:
>
>>> On Nov 9, 2019, at 10:02 AM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:
>>>
>>> Thanks Marc!
>>>
>>> Just to be clear, the pause times are not currently causing an issue for me.
>>>
>>> There is a gerbil however who has been investigating pause times and found a pathological program that allocates u8vectors and stuffs them into a hash table. In that case, there is an accumulation of memory, leading to pause times of 500-600ms, while racket averages 20ms (with a peak of about 170ms) on the equivalent program. This is clearly an artifact of generational gc, but it always vexes me when racket seemingly does better at something :p
>>>
>>> -- vyzo
>>
>> It would be good to investigate this further because the behaviour you describe is not the one I would have expected, assuming the u8vectors are not small (i.e. >= 2033 bytes on a 64 bit machine).  That’s because non-small objects are allocated as still objects and aren’t moved by the GC and the content isn’t even looked at by the GC (so a generational GC will not help).
>>
>> Perhaps the racket code is not “equivalent”.  What is the total run time of the racket code compared to Gambit?  Generational GC may reduce the time to do garbage collection, but slow down other operations (such as assignments).
>>
>> Concerning the parallel GC, there is a good 2x performance boost for a heap that contains a list of 10 million u8vectors of length 10 (see attached code):
>>
>> Without the parallel GC the pause time per GC:
>> (real-time . .48944687843322754)
>> (real-time . .35641002655029297)
>> (real-time . .3395528793334961)
>>
>> With the parallel GC (./configure --enable-multiple-threaded-vms) the pause time per GC:
>> (real-time . .23423385620117188)
>> (real-time . .17566204071044922)
>> (real-time . .1841750144958496)
>>
>> Marc
>>
>> (declare (standard-bindings) (extended-bindings) (not safe))
>>
>> (define obj
>>   (let loop ((i 10000000) (result '()))
>>     (if (fx> i 0)
>>         (loop (fx- i 1)
>>               (let ((vect (make-u8vector 10)))
>>                 (cons vect
>>                       result)))
>>         result)))
>>
>> (pp (assoc 'real-time (##exec-stats ##gc)))
>> (pp (assoc 'real-time (##exec-stats ##gc)))
>> (pp (assoc 'real-time (##exec-stats ##gc)))
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.iro.umontreal.ca/pipermail/gambit-list/attachments/20191109/b5719fce/attachment.htm>


More information about the Gambit-list mailing list