-
Notifications
You must be signed in to change notification settings - Fork 5.4k
YJIT: Add compilation log #11818
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
YJIT: Add compilation log #11818
Conversation
dabc7bc
to
5e14be1
Compare
In the commit log, at the beginning of "Directory Argument" and "File Argument",
"falg" looks like a typo. |
f824a11
to
8b5e2bf
Compare
Thank you, @nobu. I've updated the PR description. I grepped the code and the commit log and couldn't find that particular typo anywhere else in this PR. |
Testing out the compilation log output:
What does it mean when a method shows up multiple times in a row? Are we compiling the same entry point several times in a row? In this case:
Does 2 levels mean chain depth 2? In which case, could we print |
It's the depth of the Ruby block. It's just how CRuby labels block ISEQs.
That Ruby code seems to have multiple basic blocks in the same line. If we want to clarify that, we could print the index to the instruction of an ISEQ. |
Ok I got confused because I thought block (2 levels) was a reference to a YJIT block with chain depth 2, but it's actually a Ruby block. Might be good to print the iseq index as Kokubun suggests. |
I took the log format from what we we're using in diassembly dumps. It doesn't need to be in this PR, but should we update as well? |
I don't think it's worth modifying CRuby (or deviating from what it uses) for it. It's CRuby's common format, which is also used in backtraces. |
2f9b591
to
a36b7d2
Compare
I'm understanding this to mean that we don't want to modify the format in |
a36b7d2
to
3f58773
Compare
This comment has been minimized.
This comment has been minimized.
Since I'm personally used to adding stuff inside parens like
|
0b5eab6
to
6498c73
Compare
6498c73
to
6b31ca8
Compare
…ompilation log. The compilation log will be populated as compilation events occur. If a directory is supplied, then a filename based on the PID will be used as the write target. If a file name is supplied instead, the log will be written to that file.
… sent to a file. Previously, the two modes were treated as being exclusive of one another. However, it could be beneficial to log all events to a file while also allowing for direct access of the last N events via `RubyVM::YJIT.compilation_log`.
…thout an argument.
… references that may GC.
…lim down the log.
… in-memory log without printing it. Co-authored-by: Randy Stauner <[email protected]>
6b31ca8
to
5f81f1b
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nicely done Kevin 🙏
Co-authored-by: Randy Stauner <[email protected]>
Head branch was pushed to by a user without write access
676210d
to
a76d27c
Compare
This PR introduces a new compilation log to help gain insights into how YJIT is optimizing an application. Finding the correct amount information to display for a given context is a bit hard. Some folks may want an entire log while others are only interested in the most recent events. This PR tries to strike a balance by offering a few different operating modes.
Command-Line Flag
This PR introduces the
--yjit-log
flag, which functions like a hybrid of--yjit-stats
and--yjit-dump-disasm
. The presence of an optional argument will dictate where the compilation log is written.No Argument
--yjit-log
without an argument will print the compilation log to$stderr
when the application exits. Events are buffered until the application exit in order to avoid mixing messages up with other output from the application. To constrain the total amount of memory used, a circular buffer is used to store the logged events. Only the entries present in the buffer at the time the application exits will be printed.--yjit-compilation-log
without an argument will print the compilation log to$stderr
as events occur. It will be logically equivalent to running with--yjit-compilation-log=/dev/stderr
, just without file truncation. Events will still be collected in the internal circular buffer for programmatic access.Quiet Argument
--yjit-log=quiet
will activate the log but not print or write it anywhere. The log contents will be accessible in code viaRubyVM::YJIT.log
. A circular buffer is used to store the log entries to constrain growth. Consequently, only the last N=1024 entries are available. Reading the log will remove all stored values, freeing up space for new entries.Directory Argument
The
--yjit-log
flag can take an optional argument pointing at a directory (e.g.,--yjit-log=$PWD
). In this case, it functions very similarly to--yjit-dump-disasm
: a new file namedyjit_$PID.log
will be created in the specified directory and all compilation events will be logged to that file. In this way, one directory can capture output from multiple processes. NB: the directory must exist when starting Ruby, otherwise the argument will be treated as if it were a file name (see below).Unlike the case where output is sent to
$stderr
, all events will be written to the file as they occur. The circular buffer will still be populated in the event you wish to work with the log in memory (accessible viaRubyVM::YJIT.compilation_log
).File Argument
The
--yjit-log
flag can take an optional argument pointing at a file (e.g.,--yjit-log=my_compilation.log
). This works very similarly to the directory argument case, except instead of a file name being generated, it will use the one supplied.The file will be created when the process starts. If the file already exists, it will be truncated. This supports a nice workflow whereby you can re-run the same
ruby
invocation repeatedly and avoid mixing results for multiple runs. Moreover, since the file is truncated rather than re-created, you cantail -f my_compilation.log
without having to restart thetail
command each time you re-runruby
.Programmatic Control
In addition to the command-line flag, you can enable or disable the compilation log via
RubyVM::YJIT.enable
and thecompilation_log
keyword argument. It is designed to function similarly to the programmatic control of YJIT's runtime stats. Principally, it accepts three values:false
: Don't enable the compilation log (more of a placeholder since you can't call this method again to disable)true
: Enable the compilation log and print it to$stderr
at exit:quiet
: Enable the compilation log, but do not write or print itNB: There is not an option to enable the log and write to a file. If you want to operate in that mode you should supply the
--yjit-log
argument.Programmatic Access
Check if Enabled
You can check if the YJIT compilation log is currently enabled with
RubyVM::YJIT.log_enabled?
.Reading the Log
The last N=1024 compilation log entries are accessible via
RubyVM::YJIT.log
. The output has the typeArray[Tuple[Time, String]]
, providing the log is enabled. If it is not, the value will benil
to help differentiate from a log where no compilation has occurred yet (i.e., an empty array) . The log is automatically cleared when accessed so subsequent reads will be guaranteed to contain new events.