[TSan] Unlocking MonoStats, GCStats and JITGCStats #5381

cherusker · 2017-08-15T20:16:50Z

As the Unlocked* () functions have been approved in #5310, I used them to blacklist some of the most obvious counting variables whose data races should be ignored: MonoStats, GCStats and JITGCStats (more to follow if approved).

Please find details about all changes within the commit messages.

dnfclas · 2017-08-15T20:20:18Z

@cherusker,
Thanks for having already signed the Contribution License Agreement. Your agreement was validated by .NET Foundation. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

lambdageek · 2017-08-15T20:35:17Z

mono/metadata/sgen-mono.c

@BrzVlad is the size arg of sgen_client_degraded_allocation not useful?

Anything starting with sgen_client_ is part of the SGen API, so this is a breaking change.

@alexrp I don't think these are meant to be an official API and they were changed in the past. Are there other runtimes out there that actually use sgen in the first place ?

I guess @schani would know. He did the whole SGen separation effort for a use case I can't remember.

Changing this is fine.

alexrp · 2017-08-15T21:38:10Z

mono/utils/unlocked.h

Please split the changes to this file into a separate commit.

@alexrp sure; done!

cherusker · 2017-08-15T22:46:10Z

mono/sgen/sgen-gc.c

@luhenry can we compare with == GENERATION_NURSERY here?

yes you can. also, you can use Interlocked here.

vargaz · 2017-08-15T23:07:48Z

A lot of the counters are in somewhat perf sensitive code, you don't want an atomic operation every time an object etc. is created for example. The ones in the JIT are ok.

cherusker · 2017-08-15T23:23:58Z

@vargaz the Unlocked* () functions have absolutely no performance impact (if compiled without the TSan setup); if, however, the affected JIT functions are not as performance critical as the rest - should we / can we use the Interlocked* () functions there instead of the Unlocked* () ones?

vargaz · 2017-08-15T23:26:09Z

The non-perf critical ones can use the simple Interlocked functions.

lewurm

I annotated all sites that I think aren't performance critical and thus can use Interlocked. In general, I think most of the stuff in your PR isn't perf-critical, except the JIT info lookup and IMT machinery. @vargaz can you have a look again and check if my assessment is correct?

lewurm · 2017-08-17T18:14:46Z

mono/metadata/boehm-gc.c

can be Interlocked

lewurm · 2017-08-17T18:14:59Z

mono/metadata/boehm-gc.c

can be Interlocked

@vargaz @lewurm can I convert the gc_stats.*_gc_time_* values (gc-internal-agnostic.h) to gint64? The reason behind this: An InterlockedAdd64 () function exists already, however, I would have to "invent" InterlockedAddUnsigned64 (). Furthermore, @vargaz told me to simply use signed types if there is no explicit reason for using unsigned ones (#5299) and 9,223,372,036,854,775,807 ns are still almost 300 years ... do we really need 600 years?

should be fine. also, mono_100ns_ticks () returns a gint64.

lewurm · 2017-08-17T18:15:19Z

mono/metadata/loader.c

can be Interlocked

lewurm · 2017-08-17T18:18:16Z

mono/metadata/sgen-mono.c

everything can use Interlocked here

lewurm · 2017-08-17T18:18:29Z

mono/mini/driver.c

can be Interlocked

lewurm · 2017-08-17T18:23:19Z

mono/sgen/sgen-gc.c

can be Interlocked

lewurm · 2017-08-17T18:23:23Z

mono/sgen/sgen-gc.c

can be Interlocked

lewurm · 2017-08-17T18:23:27Z

mono/sgen/sgen-gc.c

can be Interlocked

lewurm · 2017-08-17T18:23:31Z

mono/sgen/sgen-gc.c

can be Interlocked

lewurm · 2017-08-17T18:25:26Z

mono/sgen/sgen-gc.c

yes you can. also, you can use Interlocked here.

- add functions for upcoming commits

- add `InterlockedWriteBool ()` which will be used in an upcoming commit

- change field types from `size_t` to `gint32` - GLib types are preferably used throughout Mono - `Interlocked* ()` functions exist for `gint32` already - `gint32` is preferred to `gint64` when looking at 32-bit machines - the extra 2,147,483,648 (0 <= `size_t` <= 4,294,967,295; -2,147,483,648 <= `gint32` <= 2,147,483,647) don't seem to matter when looking at the average values of these variables - remove unused fields; some fields have been moved to GCStats (`*_gc_*`) - use `G_*_FORMAT` to correctly print `g*` types

- `minor_gc_count` and `major_gc_count`: use `gint32` as the `binary_protocol_collection* ()` and `sgen_debug_dump_heap ()` functions cast these values to `int` anyways - `*_gc_time_*`: use `gint64` since 300 years seem like a fair amount of time and `Interlocked*64 ()` can be used - adjust the logic of `sgen_gc_collection_count ()` to enhance its readability - parameter `size` of `sgen_client_degraded_allocation` is unused and can be removed - use `Interlocked* ()` for static variables of `sgen_client_degraded_allocation ()` - use `G_*_FORMAT` to correctly print `g*` types

- change the field types from `int` to `gint32`

cherusker · 2017-08-18T19:43:13Z

mono/utils/atomic.h

+/* The following functions cannot be found on any platform, and thus they can be declared without further existence checks */
+
+static inline void
+InterlockedWriteBool (volatile gboolean *dest, gboolean val)


@BrzVlad @alexrp is it OK to put this function there?

cherusker · 2017-08-18T19:52:37Z

The most recent update changed a lot of functions from Unlocked* () to Interlocked* () and also changed the types of MonoStats variables from gsize to gint32. The reason for that is explained in the commit message: mainly it's about Interlocked* () being defined for gint32 but not for gsize. If this is unwanted, however, I can either

go back to gsize and add Interlocked*Size () methods to atomics.h or
we can go with gint64.

vargaz · 2017-08-18T20:18:46Z

I think we should use either gint32 or gint64, depending on the expected values.

cherusker · 2017-08-18T21:44:42Z

@vargaz I have seen these values going into the low millions but always staying far away from the billions with class_vtable_size usually leading. After all, I only reduced the capacity by the factor 2 (no log reduction or worse) - so generally it should not matter. Anyways, gint64 would be safer of course if it's OK to use it for performance-critical code thinking about the overhead of 64-bit operations on 32-bit machines JUST for monitoring. @kumpera do you have any thoughts on this?

vargaz · 2017-08-18T21:45:22Z

I think gint64 is only needed for things like amount of GC memory allocated.

lewurm · 2017-08-23T15:53:32Z

@monojenkins squash

[TSan] Unlocking MonoStats, GCStats and JITGCStats As the `Unlocked* ()` functions have been approved in mono/mono#5310, I used them to blacklist some of the most obvious counting variables whose data races should be ignored: `MonoStats`, `GCStats` and `JITGCStats` (more to follow if approved). Please find details about all changes within the commit messages. Commit migrated from mono/mono@787faec

cherusker requested review from BrzVlad, kumpera, lewurm and vargaz as code owners August 15, 2017 20:16

dnfclas added the cla-already-signed label Aug 15, 2017

lambdageek reviewed Aug 15, 2017

View reviewed changes

alexrp reviewed Aug 15, 2017

View reviewed changes

cherusker force-pushed the cherusker-2017-08-15-spread-unlocked branch from 4e37630 to 435c428 Compare August 15, 2017 22:32

cherusker commented Aug 15, 2017

View reviewed changes

lewurm reviewed Aug 17, 2017

View reviewed changes

vargaz approved these changes Aug 17, 2017

View reviewed changes

cherusker added 5 commits August 18, 2017 20:51

Update unlocked.h

49546ac

- add functions for upcoming commits

Update atomic.h

ccceee2

- add `InterlockedWriteBool ()` which will be used in an upcoming commit

Tell TSan to ignore JITGCStats

441a6f6

- change the field types from `int` to `gint32`

cherusker force-pushed the cherusker-2017-08-15-spread-unlocked branch from 435c428 to 441a6f6 Compare August 18, 2017 19:39

cherusker commented Aug 18, 2017

View reviewed changes

lewurm approved these changes Aug 23, 2017

View reviewed changes

monojenkins merged commit 787faec into mono:master Aug 23, 2017

cherusker deleted the cherusker-2017-08-15-spread-unlocked branch August 23, 2017 17:58

[TSan] Unlocking MonoStats, GCStats and JITGCStats #5381

[TSan] Unlocking MonoStats, GCStats and JITGCStats #5381

Uh oh!

Conversation

cherusker commented Aug 15, 2017

Uh oh!

dnfclas commented Aug 15, 2017

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vargaz commented Aug 15, 2017

Uh oh!

cherusker commented Aug 15, 2017

Uh oh!

vargaz commented Aug 15, 2017

Uh oh!

lewurm left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cherusker Aug 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cherusker Aug 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cherusker commented Aug 18, 2017

Uh oh!

vargaz commented Aug 18, 2017

Uh oh!

cherusker commented Aug 18, 2017

Uh oh!

vargaz commented Aug 18, 2017

Uh oh!

lewurm commented Aug 23, 2017

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

cherusker Aug 18, 2017 •

edited

Loading

cherusker Aug 18, 2017 •

edited

Loading