-
Notifications
You must be signed in to change notification settings - Fork 3k
Fix longstanding errors in the compiler's alias analysis pass #7448
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
CT Test Results 2 files 296 suites 14m 36s ⏱️ Results for commit c389665. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts
// Erlang/OTP Github Action Bot |
|
Tweak commit subject for 5175353. |
|
Thanks for this PR, but it introduces a compiler crash on: f() ->
erlang:element(ok, length(erlang:pre_loaded())).erlc crashes with |
|
The bug found by @RobinMorisset should now be fixed. |
|
The build failures are due to |
|
Thanks for the quick fix! f() ->
[ 0 || _V1 <- erlang:memory(),
{ maybe
error ?= _V1,
ok
end,
maybe
{<<_>>} ?= _V1,
ok
end } ].Without this PR it compiles fine, but with it erlc fails with: |
|
Another day, another fix :) @RobinMorisset, how much CPU-time was required to find the latest crash? I'm starting to think I should set up my own fuzzing instance. Is it worthwhile to run on a single PC overnight, or is a cluster/cloud required to get results? |
|
It took a few hours on a moderately beefy server (on par with a high-end gaming PC I'd say), so it would probably have taken a day or two of running on the average laptop. |
Then I think I'll leave the fuzzing to you :) |
|
The fuzzer did not find any new bug on this PR for a couple of days, so it looks good to me. |
When a value is extracted from a map and the map dies, the extracted value should inherit the status of the map. If the map does not die, both the map and the extracted value should be aliased. This patch corrects an error in the alias analysis pass which failed to propagate a status of 'aliased' from the map to the extracted value. It also unifies the handling of the BIF `map_get` and the instruction `get_map_element`. The patch also adds test-cases to check that both the BIF and the instruction behaves the same.
When the BIF `element` was used to extract an element from a tuple, the result is wrong and does not not match the result of using the `get_tuple_element` instruction. This patch corrects the error by using the same code as for the instruction to handle the BIF.
When the BIFs `hd/1` and `tl/1` are used to extract an element from a pair, the results are wrong and do not match the result of using the `get_hd` and `get_tl` instructions. This patch corrects the error by using the same code as for instructions to handle the BIFs.
When elements are extracted from pairs and tuples, the extracted element forms an alias to the element in the original aggregate. But as long as the reference to the original is only used to extract other elements and the original reference is not use for anything else, it does not lead to any aliasing that is an issue for destructive updates. This patch enhances the alias analysis to account for extracted elements. This is done by extending `aa_get_status/2` to consider a reference from which elements has been extracted as aliased. Likewise `aa_set_status/3` now updates the status of all extracted elements if the original reference becomes aliased. Correspondingly `aa_tuple_extraction/2` and `aa_pair_extraction/4` are modified to disregard extracted elements as long as the element being extracted has not already been extracted. As this enhancement will let the private-append pass explore new execution paths which were ignored previously due to aliasing, the value tracking is updated to handle mismatched element types.
Information about the aliasing state of call arguments used to be
accumulated in the local, per function, alias status map and then
merged into the global #aas{} when the function was done.
This patch changes the handling of the aliasing state of call
arguments to update the `#aas{}` directly. The change requires the
conditions for determining when a single iteration has led to changes,
to also include `#aas.call_args`.
Functionally this patch leads to no changes in the output of the
compiler. It is the first step in switching the alias analysis from a
per-function unified alias status map to a representation that can
account for different execution paths within the function.
The alias analysis pass uses an algorithm inspired Kotzmann and
Mössenböck's 2005 algorithm for Escape Analysis. Unfortunately the
algorithm completely fails to identify some cases of aliasing due to
they way it tracks aliasing and updates annotations.
The alias analysis is performed by traversing the functions in the
module and their code. For each operation the uniqueness and alias
status are updated and annotations are updated. The unique/aliased
status is maintained in a map which maps a variable to a either a
status or another variable. Thus constructing equivalent sets in the
same way as Kotzmann and Mössenböck. At call sites aliasing
information is fed back to the callee and the alias analysis is repeated
for the callee. For loops this works correctly, but for non-loops this
can lead to bad aliased/unique verdicts. Consider the following
example:
1: tuple_element_aliasing() ->
2: T = make_empty_binary_tuple(),
3: X = element(1, T),
4: Z = <<X/binary,1:8>>,
5: {Z, T}.
6: make_empty_binary_tuple() ->
7: {<<>>, <<>>}.
At line 4, the algorithm will mark X as unique as there is no feedback
from line 5 where a reference to the tuple from which X is extracted
is captured in the newly constructed tuple.
The obvious solution of waiting to update alias/unique annotations
until all basic blocks of a function have been processed, avoids the
error described above, but also disables many private-append
optimizations where the broken algorithm was correct. The main reason
for this, is that the algorithm does not consider flow-control
relationships between blocks in a function, to it, all blocks are
executed even if flow control guarantees that only one of two blocks
will ever be executed.
This patch greatly strengthens the analysis done by aliasing analysis
by accounting for flow-control within functions. It also changes the
data structure used to describe the alias status of variables in order
to support incorporating flow-control information in the per function
analysis.
In short, this patch does two things, neither of which can be done in
isolation without breaking the test suite:
1) The data structure used to describe the alias status of variables,
called a Sharing State, is changed to track three aspects:
- A variable is either unique, aliased.
- A unique variable can be derived, that is extracted from a term
or constructed from other variables.
- The relationship between the source and a derived value is
tracked so that it is possible to find the parent of a derived
value and likewise find all values that are derived from a
value.
2) The basic blocks of a function are now traversed reverse post
order. When the end of a block is reached, the current sharing
state is propagated to the block's successors by merging the
sharing state of all the successor block's predecessors' sharing
states.
When all blocks have been visited and the sharing states at the end
of each block are known, information about aliased variables are
propagated along the edges of the execution graph during a post
order traversal of the basic blocks.
compiler: Change data structure describing the alias status of variables
Change the data structure describing the alias status of
variables. Instead of a map, where different keys represented
different aspects of a variable, this patch maps a variable to a
single record describing all aspects of the variable. The change
greatly speeds up the alias analysis by reducing the number of visited
entries when merging two alias states while also reducing the
complexity of the merging logic.
|
Refreshed the patch onto today's maint. It includes a complete rewrite of the top "compiler: Correct errors in alias analysis pass" commit. The new version has better performance, is easier to understand and comes with asserts checking its invariants. In short, it is what the first version should have been :) |
This is a set of patches which fixes a number of errors in the compiler's alias analysis pass which have been there since the code was first merged. The final commit is unfortunately large, but I have been unable to create a smaller patch, which doesn't disable previously working and correct test cases.