[gambit-list] Dumping the heap

Dimitris Vyzovitis vyzo at hackzen.org
Fri Feb 2 08:47:31 EST 2018


note: you don't have to implement on-all-processors, i can do that in
gerbil stdlib;
but pinning needs to be exposed in some way (## procedures are fine).

-- vyzo

On Fri, Feb 2, 2018 at 3:42 PM, Dimitris Vyzovitis <vyzo at hackzen.org> wrote:

> pinning, if exposed, should be sufficient to implement it purely in
> userland.
>
> it would be immediately useful for my heap dumper -- i could use it to get
> a
> vector of stills from all processors with the
> count-still-objects/get-still-objects
> procedures before starting the walk and use that to ensure that all stills
> (at the
> beginning of the walk) are accounted for.
>
> it would also be useful for implementing a parallel dispatch primitive
> that utilizes
> all cores maximally.  say you have a parallel algorithm that you want to
> decompose
> into per core tasks, that could be accomplished with on-all-processors (or
> a similar
> primitive based on pinning).  and it doesn't have to be a compute
> algorithm, i/o could
> benefit too.
>
> -- vyzo
>
> On Fri, Feb 2, 2018 at 3:32 PM, Marc Feeley <feeley at iro.umontreal.ca>
> wrote:
>
>> The SMP scheduler support “pinning” threads to processors, so perhaps
>> this is implementable.  However… why do you need this?  I don’t like
>> exposing the processor concept or pinning, which are low-level concepts.
>>
>> Marc
>>
>>
>>
>> > On Feb 2, 2018, at 8:07 AM, Dimitris Vyzovitis <vyzo at hackzen.org>
>> wrote:
>> >
>> > perhaps the "don't switch" semantics are too much.
>> > a simpler general purpose primitive would be an `on-all-processors`
>> that spawns
>> > a thread on each processor to execute the thunk and completes when all
>> thunks
>> > have completed.
>> >
>> > that's likely implementable without any deep support from the runtime.
>> >
>> > -- vyzo
>> >
>> >
>> > On Fri, Feb 2, 2018 at 3:00 PM, Dimitris Vyzovitis <vyzo at hackzen.org>
>> wrote:
>> > well, perhaps we can think about the right primitive for Scheme level
>> operations.
>> > the semantics could be something like "execute this thunk on all
>> processors, and
>> > don't do any switches until it has finished executing".
>> >
>> > -- vyzo
>> >
>> >
>> > On Fri, Feb 2, 2018 at 2:57 PM, Marc Feeley <feeley at iro.umontreal.ca>
>> wrote:
>> > on_all_processors was designed for the lowest-level of the runtime
>> system, I don’t think it is possible for the operation to be in Scheme
>> (I’ll have to thinks about what the constraints are on the operation).
>> >
>> > Marc
>> >
>> >
>> >
>> > > On Feb 2, 2018, at 7:49 AM, Dimitris Vyzovitis <vyzo at hackzen.org>
>> wrote:
>> > >
>> > > it would be nice to have a primitive to do this for Scheme procedures!
>> > > Something like (on-all-processors thunk) would be awesome.
>> > >
>> > > -- vyzo
>> > >
>> > > On Fri, Feb 2, 2018 at 2:41 PM, Marc Feeley <feeley at iro.umontreal.ca>
>> wrote:
>> > > Yes each processor has its own still_objs list and to account for all
>> still objects you must iterate over the processors.  In order to avoid
>> modification of the still_objs lists while doing this the best approach is
>> to use the barrier operation mechanism.  That way all processors (but one)
>> will be idle while iterating (or you could have all processors cooperate).
>> This is done with the “on_all_processors” function.  For an example, check
>> out ___garbage_collect or ___fdset_resize in lib/setup.c .
>> > >
>> > > Marc
>> > >
>> > >
>> > >
>> > > > On Feb 2, 2018, at 6:23 AM, Dimitris Vyzovitis <vyzo at hackzen.org>
>> wrote:
>> > > >
>> > > > Relevant code for accounting still objects:
>> > > > https://gist.github.com/vyzo/ab4219382c0870779991d4c701921d2c
>> > > >
>> > > > The limitation is that the still_objs_ is per processor, and not
>> vm-wide.
>> > > > Does that mean we would have to crawl all processors in SMP?
>> > > >
>> > > > -- vyzo
>> > > >
>> > > >
>> > > > On Thu, Feb 1, 2018 at 3:16 PM, Marc Feeley <
>> feeley at iro.umontreal.ca> wrote:
>> > > > > On Feb 1, 2018, at 8:06 AM, Dimitris Vyzovitis <vyzo at hackzen.org>
>> wrote:
>> > > > >
>> > > > > thanks Guillaume!
>> > > > >
>> > > > > this is a great start for me -- i am helping fare debug a memory
>> leak, and it's really hard to identify
>> > > > > without dumping the heap to see what kind of object is leaking.
>> > > >
>> > > > For your information I discovered a few memory leaks with the
>> networking functions.  They were due to “sockaddr” structures being
>> converted to “still” Scheme objects with a reference count = 1, but the
>> reference count was never decremented (with ___release_scmobj).  This has
>> been fixed in the recent UDP commit.
>> > > >
>> > > > I believe that this kind of situation might exist in other places
>> in the runtime system.  So it might be useful to debug this to have a
>> function that returns a list of all the “still” Scheme objects that have a
>> reference count != 0.  This should be easy to write… the GC maintains a
>> list of the still objects in the C variable “still_objs”.
>> > > >
>> > > > So the idea would be to check at the end of a program if there are
>> any still objects with non-zero ref counts.
>> > > >
>> > > > Marc
>> > > >
>> > > >
>> > >
>> > >
>> >
>> >
>> >
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://webmail.iro.umontreal.ca/pipermail/gambit-list/attachments/20180202/dc22d95f/attachment-0001.html 


More information about the Gambit-list mailing list