Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
65 commits
Select commit Hold shift + click to select a range
53ce76f
[profiler] Only emit code chunk events when MONO_PROFILE_JIT_COMPILAT…
Sep 9, 2016
3c273ab
[profiler] Add a new internal API to switch the runtime to profiling …
Sep 9, 2016
95e8c6f
[sgen] Add a new variant of the managed allocator that is used when p…
Sep 9, 2016
6c60be2
[profiler] Add some documentation for event flags.
Sep 9, 2016
1b950f0
[profiler] Add counters for all event types.
Sep 9, 2016
6c730c5
[profiler] Fix the code that synthesizes thread end events.
Sep 9, 2016
2bcc30f
[profiler] Don't emit a perf counters sample event if none are regist…
Sep 9, 2016
d12e7bb
[profiler] Simplify counters initialization.
Sep 9, 2016
4703e01
[profiler] Don't leak counter data structures on shutdown.
Sep 9, 2016
6bb0d1a
[profiler] Fix the test-traces.exe test after the change to managed a…
Sep 9, 2016
54ba5a2
[profiler] Mark mono_profiler_get_events () as MONO_RT_EXTERNAL_ONLY.
Sep 9, 2016
0abe70c
[profiler] Lift profiler limitation in mono_class_get_allocation_ftn ().
Sep 9, 2016
e9eed48
[profiler] Remove some indirection in setting up GC events.
Sep 9, 2016
174e554
[profiler] Allow changing sampling parameters at runtime.
Sep 9, 2016
2a8c6d0
[profiler] Expose sampling mode/frequency variables directly to the r…
Sep 9, 2016
461d0aa
[profiler] Use the signal shutdown logic for the sampling thread for …
Sep 9, 2016
2f45c5f
[man] Update mprof-report.1 to indicate the correct default maxframes…
Sep 9, 2016
2a475ca
[profiler] Remove the countersonly option.
Sep 9, 2016
c97d2e7
[profiler] Document and fix the onlycoverage option.
Sep 9, 2016
4082074
[profiler] Remove the long-obsolete hsmode option.
Sep 9, 2016
80a2fba
[utils] Move MONO_DEPRECATED to mono-publib.h and make it imply MONO_…
Sep 9, 2016
6c52a10
[profiler] Introduce a new profiler API.
Sep 9, 2016
29fd30e
[profiler] Fire method exceptional exit events for MONO_PROFILE_ENTER…
Sep 9, 2016
3153441
[profiler] Add MONO_PROFILE_GC_HANDLES and use it for GC handle event…
Sep 9, 2016
66c74c3
[profiler] Only emit heap root events when we do a heapshot.
Sep 9, 2016
eeb2d37
[sgen] Remove bogus mono_profiler_events assert.
Sep 9, 2016
ccc69b1
[profiler] Consistently check mono_profiler_events before invoking pr…
Sep 9, 2016
f227edd
[profiler] Remove utils.c/utils.h, move the code to proflog.c/decode.…
Sep 9, 2016
bbbfbd5
[profiler] Rename proflog.{c,h} to mono-profiler-log.{c,h} for consis…
Sep 10, 2016
2294b3b
[profiler] Rename decode.c to mprof-report.c.
Sep 10, 2016
8e04f49
[ci] Bump the timeout of the profiler stress test suite to 24 hours.
Sep 10, 2016
e839295
[acceptance-tests] Improve the profiler-stress test runner.
Sep 10, 2016
ef69ed4
[sgen] Always report object moves on the GC thread.
Sep 10, 2016
fce0804
[profiler] Add an extra assert.
Sep 12, 2016
358304e
[profiler] Don't flush empty log buffers.
Sep 12, 2016
0bf163e
[profiler] Fix several race conditions in accessing MonoProfilerThrea…
Sep 12, 2016
b3e66dc
[profiler] Add a missing deinit_thread () call to the helper thread.
Sep 12, 2016
0bcf83c
[profiler] Encapsulate some common logic in the ENTER_LOG/EXIT_LOG ma…
Sep 12, 2016
cc47406
[profiler] Always enable the helper thread.
Sep 12, 2016
f703d7d
[profiler] assert () -> g_assert ().
Sep 12, 2016
f093410
[profiler] Remove some broken, commented-out code from register_metho…
Sep 12, 2016
cf42d7f
[profiler] Remove mono_threads_attach_tools_thread () declaration.
Sep 12, 2016
c23c421
[profiler] Clean up the #include mess in mono-profiler-log.c.
Sep 12, 2016
a6ead88
[profiler] Remove the ign_res () hack.
Sep 12, 2016
63231c4
[profiler] Add messages to most assertions.
Sep 12, 2016
1101b43
[profiler] Replace a check with an assert in coverage_filter ().
Sep 12, 2016
c7e9f62
[profiler] Fix the way the profiler frees hazardous data on shutdown.
Sep 12, 2016
5fb8a51
[profiler] Remove the need to pass a MonoProfiler argument to some fu…
Sep 12, 2016
63b5cb2
[profiler] Process requests in method_jitted ().
Sep 12, 2016
5b5dfda
[profiler] Clean up and improve the helper thread code.
Sep 13, 2016
89cd243
[profiler] Disable GC move events when allocation events are disabled.
Sep 13, 2016
8a467c7
[utils/lfa] Add a missing #include to fix the profiler build.
Sep 13, 2016
419d9f1
[profiler] Drop support for perf events on Linux.
Sep 14, 2016
86c25c2
[profiler] Bump profiler version to v1.1.
Sep 14, 2016
c51f1db
[profiler] Set up callbacks unconditionally.
Sep 14, 2016
9a01e77
[profiler] Access in_shutdown atomically.
Sep 14, 2016
b8ef38c
[utils/threads] Export all native thread functions.
Sep 14, 2016
5dc9108
[utils/threads] Add and export a mono_native_thread_join () function.
Sep 14, 2016
b83a537
[profiler] Use the native thread wrapper functions to manage threads.
Sep 14, 2016
22d6698
[profiler] Warn if we can't detect the CPU count.
Sep 14, 2016
a867af5
[profiler] Use eglib alloc/free functions everywhere.
Sep 14, 2016
6c36051
[profiler] Address a couple of FIXMEs in the counters sampling code.
Sep 14, 2016
d1fe4ae
[profiler] Add a nocounters option.
Sep 14, 2016
5e1e678
[profiler] Don't emit enter/leave events in onlycoverage mode.
Sep 14, 2016
474e374
[profiler] Fix a regression that caused all buffers to have a zero th…
Sep 16, 2016
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion acceptance-tests/profiler-stress.mk
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@ SYS_REFS = \
System.Data.dll \
System.Runtime.Serialization.dll \
System.Xml.dll \
System.Xml.Linq.dll
System.Xml.Linq.dll \
Mono.Posix.dll

check-profiler-stress:
@$(MAKE) validate-benchmarker RESET_VERSIONS=1
Expand Down
204 changes: 187 additions & 17 deletions acceptance-tests/profiler-stress/runner.cs
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.Threading;
using System.Xml;
using Mono.Unix.Native;
using Newtonsoft.Json;

// Shut up CLS compliance warnings from Json.NET.
Expand All @@ -10,7 +17,8 @@
namespace Mono.Profiling.Tests.Stress {

// https://github.com/xamarin/benchmarker/blob/master/tools/libdbmodel/Benchmark.cs
class Benchmark {
sealed class Benchmark {

public string Name { get; set; }
public string TestDirectory { get; set; }
public bool OnlyExplicit { get; set; }
Expand All @@ -24,8 +32,24 @@ public static Benchmark Load (string file)
}
}

sealed class TestResult {

public Benchmark Benchmark { get; set; }
public ProcessStartInfo StartInfo { get; set; }
public Stopwatch Stopwatch { get; set; } = new Stopwatch ();
public int? ExitCode { get; set; }
public StringBuilder StandardOutput { get; set; } = new StringBuilder ();
public StringBuilder StandardError { get; set; } = new StringBuilder ();
}

static class Program {

static readonly TimeSpan _timeout = TimeSpan.FromHours (6);

static string FilterInvalidXmlChars (string text) {
return Regex.Replace (text, @"[^\x09\x0A\x0D\x20-\uD7FF\uE000-\uFFFD\u10000-\u10FFFF]", string.Empty);
}

static int Main ()
{
var depDir = Path.Combine ("..", "external", "benchmarker");
Expand All @@ -44,8 +68,7 @@ static int Main ()
var rand = new Random ();
var cpus = Environment.ProcessorCount;

var successes = 0;
var failures = 0;
var results = new List<TestResult> (benchmarks.Length);

var sw = Stopwatch.StartNew ();

Expand All @@ -66,6 +89,8 @@ static int Main ()
WorkingDirectory = Path.Combine (testDir, bench.TestDirectory),
FileName = monoPath,
Arguments = $"--debug --profile=log:{profOptions} " + string.Join (" ", bench.CommandLine),
RedirectStandardOutput = true,
RedirectStandardError = true,
};

info.EnvironmentVariables.Clear ();
Expand All @@ -77,32 +102,177 @@ static int Main ()
Console.WriteLine ($"[{sw.Elapsed.ToString ("G")}] {progress} Running {bench.Name} with profiler options: {profOptions}");
Console.ResetColor ();

var sw2 = Stopwatch.StartNew ();
var result = new TestResult {
Benchmark = bench,
StartInfo = info,
};

using (var proc = new Process ()) {
proc.StartInfo = info;

proc.OutputDataReceived += (sender, args) => {
if (args.Data != null)
result.StandardOutput.AppendLine (args.Data);
};

proc.ErrorDataReceived += (sender, args) => {
if (args.Data != null)
result.StandardError.AppendLine (args.Data);
};

result.Stopwatch.Start ();

using (var proc = Process.Start (info)) {
proc.WaitForExit ();
sw2.Stop ();
proc.Start ();

Console.WriteLine ();
proc.BeginOutputReadLine ();
proc.BeginErrorReadLine ();

if (proc.ExitCode != 0)
failures++;
else
successes++;
if (!proc.WaitForExit ((int) _timeout.TotalMilliseconds)) {
// Force a thread dump.
Syscall.kill (proc.Id, Signum.SIGQUIT);
Thread.Sleep (1000);

Console.ForegroundColor = proc.ExitCode != 0 ? ConsoleColor.Red : ConsoleColor.Green;
Console.WriteLine ($"[{sw.Elapsed.ToString ("G")}] {progress} {bench.Name} took {sw2.Elapsed.ToString ("G")} and exited with code: {proc.ExitCode}");
try {
proc.Kill ();
} catch (Exception) {
}
} else
result.ExitCode = proc.ExitCode;

result.Stopwatch.Stop ();
}

var resultStr = result.ExitCode == null ? "timed out" : $"exited with code: {result.ExitCode}";

Console.ForegroundColor = result.ExitCode != 0 ? ConsoleColor.Red : ConsoleColor.Green;
Console.WriteLine ($"[{sw.Elapsed.ToString ("G")}] {progress} {bench.Name} took {result.Stopwatch.Elapsed.ToString ("G")} and {resultStr}");
Console.ResetColor ();

if (result.ExitCode != 0) {
Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine ("===== stdout =====");
Console.ResetColor ();

Console.WriteLine (result.StandardOutput.ToString ());

Console.ForegroundColor = ConsoleColor.Red;
Console.WriteLine ("===== stderr =====");
Console.ResetColor ();

Console.WriteLine (result.StandardError.ToString ());
}

results.Add (result);
}

sw.Stop ();

Console.ForegroundColor = failures != 0 ? ConsoleColor.Red : ConsoleColor.Green;
Console.WriteLine ($"[{sw.Elapsed.ToString ("G")}] Finished with {successes}/{benchmarks.Length} passing tests");
var successes = results.Count (r => r.ExitCode == 0);
var failures = results.Count (r => r.ExitCode != null && r.ExitCode != 0);
var timeouts = results.Count (r => r.ExitCode == null);

var settings = new XmlWriterSettings {
NewLineOnAttributes = true,
Indent = true,
};

using (var writer = XmlWriter.Create ("TestResult-profiler-stress.xml", settings)) {
writer.WriteStartDocument ();
writer.WriteComment ("This file represents the results of running a test suite");

writer.WriteStartElement ("test-results");
writer.WriteAttributeString ("name", "profiler-stress-tests.dummy");
writer.WriteAttributeString ("total", results.Count.ToString ());
writer.WriteAttributeString ("failures", failures.ToString ());
writer.WriteAttributeString ("not-run", "0");
writer.WriteAttributeString ("date", DateTime.Now.ToString ("yyyy-MM-dd"));
writer.WriteAttributeString ("time", DateTime.Now.ToString ("HH:mm:ss"));

writer.WriteStartElement ("environment");
writer.WriteAttributeString ("nunit-version", "2.4.8.0");
writer.WriteAttributeString ("clr-version", Environment.Version.ToString ());
writer.WriteAttributeString ("os-version", Environment.OSVersion.ToString ());
writer.WriteAttributeString ("platform", Environment.OSVersion.Platform.ToString ());
writer.WriteAttributeString ("cwd", Environment.CurrentDirectory);
writer.WriteAttributeString ("machine-name", Environment.MachineName);
writer.WriteAttributeString ("user", Environment.UserName);
writer.WriteAttributeString ("user-domain", Environment.UserDomainName);
writer.WriteEndElement ();

writer.WriteStartElement ("culture-info");
writer.WriteAttributeString ("current-culture", CultureInfo.CurrentCulture.Name);
writer.WriteAttributeString ("current-uiculture", CultureInfo.CurrentUICulture.Name);
writer.WriteEndElement ();

writer.WriteStartElement ("test-suite");
writer.WriteAttributeString ("name", "profiler-stress-tests.dummy");
writer.WriteAttributeString ("success", (failures + timeouts == 0).ToString ());
writer.WriteAttributeString ("time", ((int) sw.Elapsed.TotalSeconds).ToString ());
writer.WriteAttributeString ("asserts", (failures + timeouts).ToString ());
writer.WriteStartElement ("results");

writer.WriteStartElement ("test-suite");
writer.WriteAttributeString ("name", "MonoTests");
writer.WriteAttributeString ("success", (failures + timeouts == 0).ToString ());
writer.WriteAttributeString ("time", ((int) sw.Elapsed.TotalSeconds).ToString ());
writer.WriteAttributeString ("asserts", (failures + timeouts).ToString ());
writer.WriteStartElement ("results");

writer.WriteStartElement ("test-suite");
writer.WriteAttributeString ("name", "profiler-stress");
writer.WriteAttributeString ("success", (failures + timeouts == 0).ToString ());
writer.WriteAttributeString ("time", ((int) sw.Elapsed.TotalSeconds).ToString ());
writer.WriteAttributeString ("asserts", (failures + timeouts).ToString ());
writer.WriteStartElement ("results");

foreach (var result in results) {
var timeoutStr = result.ExitCode == null ? "_timeout" : string.Empty;

writer.WriteStartElement ("test-case");
writer.WriteAttributeString ("name", $"MonoTests.profiler-stress.{result.Benchmark.Name}{timeoutStr}");
writer.WriteAttributeString ("executed", "True");
writer.WriteAttributeString ("success", (result.ExitCode == 0).ToString ());
writer.WriteAttributeString ("time", ((int) result.Stopwatch.Elapsed.TotalSeconds).ToString ());
writer.WriteAttributeString ("asserts", result.ExitCode == 0 ? "0" : "1");

if (result.ExitCode != 0) {
writer.WriteStartElement ("failure");

writer.WriteStartElement ("message");
writer.WriteCData (FilterInvalidXmlChars (result.StandardOutput.ToString ()));
writer.WriteEndElement ();

writer.WriteStartElement ("stack-trace");
writer.WriteCData (FilterInvalidXmlChars (result.StandardError.ToString ()));
writer.WriteEndElement ();

writer.WriteEndElement ();
}

writer.WriteEndElement ();
}

writer.WriteEndElement ();
writer.WriteEndElement ();

writer.WriteEndElement ();
writer.WriteEndElement ();

writer.WriteEndElement ();
writer.WriteEndElement ();

writer.WriteEndElement ();

writer.WriteEndDocument ();
}

var failureStr = failures + timeouts != 0 ? $" ({failures} failures, {timeouts} timeouts)" : string.Empty;

Console.ForegroundColor = failures + timeouts != 0 ? ConsoleColor.Red : ConsoleColor.Green;
Console.WriteLine ($"[{sw.Elapsed.ToString ("G")}] Finished with {successes}/{results.Count} passing tests{failureStr}");
Console.ResetColor ();

return failures;
return failures + timeouts;
}
}
}
6 changes: 0 additions & 6 deletions configure.ac
Original file line number Diff line number Diff line change
Expand Up @@ -2666,12 +2666,6 @@ fi

AC_ARG_ENABLE(bcl-opt, [ --disable-bcl-opt BCL is compiled with no optimizations (allows accurate BCL debugging)], test_bcl_opt=$enableval, test_bcl_opt=yes)

AC_ARG_ENABLE(perf-events, [ --enable-perf-events Enable using `perf` for profiling on Linux], test_perf_events=$enableval, test_perf_events=no)
if test "x$test_perf_events" = "xyes"; then
AC_DEFINE(ENABLE_PERF_EVENTS, 1, [Enable using `perf` for profiling on Linux])
AC_SUBST(ENABLE_PERF_EVENTS)
fi

AC_MSG_CHECKING([if big-arrays are to be enabled])
AC_ARG_ENABLE(big-arrays, [ --enable-big-arrays Enable the allocation and indexing of arrays greater than Int32.MaxValue], enable_big_arrays=$enableval, enable_big_arrays=no)
if test "x$enable_big_arrays" = "xyes" ; then
Expand Down
59 changes: 10 additions & 49 deletions man/mprof-report.1
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ provided by the Mono runtime and write them to a file named
\f[I]output.mlpd\f[].
When no option is specified, it is equivalent to using:
.PP
\f[B]--profile=log:calls,alloc,output=output.mlpd,maxframes=8,calldepth=100\f[]
\f[B]--profile=log:calls,alloc,output=output.mlpd,maxframes=32,calldepth=100\f[]
.PP
The following options can be used to modify this default behaviour.
Each option is separated from the next by a \f[B],\f[] character,
Expand Down Expand Up @@ -139,41 +139,16 @@ garbage collections
to the control port
.RE
.IP \[bu] 2
\f[I]sample[=TYPE[/FREQ]]\f[]: collect statistical samples of the
\f[I]sample[=FREQ]\f[]: collect statistical samples of the
program behaviour.
The default is to collect a 100 times per second (100 Hz) the
instruction pointer.
This is equivalent to the value \[lq]cycles/100\[rq].
This is equivalent to the value \[lq]100\[rq].
A value of zero for \f[I]FREQ\f[] effectively disables sampling.
On some systems, like with recent Linux kernels, it is possible to
cause the sampling to happen for other events provided by the
performance counters of the cpu.
In this case, \f[I]TYPE\f[] can be one of:
.RS 2
.IP \[bu] 2
\f[I]cycles\f[]: processor cycles
.IP \[bu] 2
\f[I]instr\f[]: executed instructions
.IP \[bu] 2
\f[I]cacherefs\f[]: cache references
.IP \[bu] 2
\f[I]cachemiss\f[]: cache misses
.IP \[bu] 2
\f[I]branches\f[]: executed branches
.IP \[bu] 2
\f[I]branchmiss\f[]: mispredicted branches
.RE
.IP \[bu] 2
\f[I]time=TIMER\f[]: use the TIMER timestamp mode.
TIMER can have the following values:
.RS 2
.IP \[bu] 2
\f[I]fast\f[]: a usually faster but possibly more inaccurate timer
.RE
.IP \[bu] 2
\f[I]maxframes=NUM\f[]: when a stack trace needs to be performed,
collect \f[I]NUM\f[] frames at the most.
The default is 8.
The default is 32.
.IP \[bu] 2
\f[I]maxsamples=NUM\f[]: stop allocating reusable sample events
once \f[I]NUM\f[] events have been allocated (a value of zero for
Expand Down Expand Up @@ -234,16 +209,15 @@ The following commands are available:
\f[I]heapshot\f[]: perform a heapshot as soon as possible
.RE
.IP \[bu] 2
\f[I]counters\f[]: sample counters values every 1 second. This allow
a really lightweight way to have insight in some of the runtime key
metrics. Counters displayed in non verbose mode are : Methods from AOT,
Methods JITted using mono JIT, Methods JITted using LLVM, Total time
spent JITting (sec), User Time, System Time, Total Time, Working Set,
Private Bytes, Virtual Bytes, Page Faults and CPU Load Average (1min,
5min and 15min).
\f[I]nocounters\f[]: disables sampling of runtime and performance
counters, which is normally done every 1 second.
.IP \[bu] 2
\f[I]coverage\f[]: collect code coverage data. This implies enabling
the \f[I]calls\f[] option.
.IP \[bu] 2
\f[I]onlycoverage\f[]: can only be used with \f[I]coverage\f[]. This
disables most other events so that the profiler mostly only collects
coverage data.
.RE
.SS Analyzing the profile data
.PP
Expand Down Expand Up @@ -274,10 +248,6 @@ with the \f[I]--maxframes=NUM\f[] option:
The stack trace info will be available if method enter/leave events
have been recorded or if stack trace collection wasn't explicitly
disabled with the \f[I]maxframes=0\f[] profiler option.
Note that the profiler will collect up to 8 frames by default at
specific events when the \f[I]nocalls\f[] option is used, so in
that case, if more stack frames are required in mprof-report, a
bigger value for maxframes when profiling must be used, too.
.PP
The \f[I]--traces\f[] option also controls the reverse reference
feature in the heapshot report: for each class it reports how many
Expand Down Expand Up @@ -487,15 +457,6 @@ option: especially if the managed heap is big, since every object
needs to be inspected.
The \f[I]MODE\f[] parameter of the \f[I]heapshot\f[] option can be
used to reduce the frequency of the heap shots.
.IP "\f[I]Reduce the timestamp overhead\f[]" 4
.Sp
On many operating systems or architectures what actually slows down
profiling is the function provided by the system to get timestamp
information.
The \f[I]time=fast\f[] profiler option can be usually used to speed
up this operation, but, depending on the system, time accounting
may have some level of approximation (though statistically the data
should be still fairly valuable).
.SS Dealing with the size of the data files
.PP
When collecting a lot of information about a profiled program, huge
Expand Down
1 change: 1 addition & 0 deletions mono/cil/cil-opcodes.xml
Original file line number Diff line number Diff line change
Expand Up @@ -320,4 +320,5 @@
<opcode name="mono_lddomain" input="Pop0" output="PushI" args="InlineNone" o1="0xF0" o2="0x19" flow="next" />
<opcode name="mono_atomic_store_i4" input="PopI+PopI" output="Push0" args="InlineI" o1="0xF0" o2="0x1A" flow="next" />
<opcode name="mono_get_last_error" input="Pop0" output="PushI" args="InlineNone" o1="0xF0" o2="0x1B" flow="next" />
<opcode name="mono_ldptr_profiler_events" input="Pop0" output="PushI" args="InlineNone" o1="0xF0" o2="0x1C" flow="next" />
</opdesc>
Loading