-
Notifications
You must be signed in to change notification settings - Fork 13.5k
[VPlan] Verify dominance for incoming values of phi-like recipes. #124838
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
@llvm/pr-subscribers-vectorizers Author: Florian Hahn (fhahn) ChangesUpdate the verifier to verify dominance for incoming values for phi-like recipes. The defining recipe must dominate the incoming block for the incoming value. There are 4 different cases to consider when retrieving the incoming block:
Full diff: https://github.com/llvm/llvm-project/pull/124838.diff 1 Files Affected:
diff --git a/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp b/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
index 0f151c897d938e..0948452e8d19cf 100644
--- a/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
+++ b/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
@@ -175,6 +175,11 @@ bool VPlanVerifier::verifyEVLRecipe(const VPInstruction &EVL) const {
});
}
+/// Return true if \p R is a VPIRInstruction wrapping a phi.
+static bool isVPIRInstructionPhi(const VPRecipeBase &R) {
+ auto *VPIRI = dyn_cast<VPIRInstruction>(&R);
+ return VPIRI && isa<PHINode>(VPIRI->getInstruction());
+}
bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) {
if (!verifyPhiRecipes(VPBB))
return false;
@@ -207,14 +212,57 @@ bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) {
for (const VPUser *U : V->users()) {
auto *UI = cast<VPRecipeBase>(U);
- // TODO: check dominance of incoming values for phis properly.
- if (!UI ||
- isa<VPHeaderPHIRecipe, VPWidenPHIRecipe, VPPredInstPHIRecipe>(UI))
+ const VPBlockBase *UserVPBB = UI->getParent();
+
+ // Verify incoming values of VPIRInstructions wrapping phis. V most
+ // dominate the end of the incoming block. The operand index of the
+ // incoming value matches the predecessor block index of the
+ // corresponding incoming block.
+ if (isVPIRInstructionPhi(*UI)) {
+ for (const auto &[Idx, Op] : enumerate(UI->operands())) {
+ if (V != Op)
+ continue;
+ const VPBlockBase *Incoming = UserVPBB->getPredecessors()[Idx];
+ if (Incoming != VPBB && !VPDT.dominates(VPBB, Incoming)) {
+ errs() << "Use before def!\n";
+ return false;
+ }
+ }
continue;
+ }
+ // Verify incoming value of various phi-like recipes.
+ if (isa<VPWidenPHIRecipe, VPHeaderPHIRecipe, VPPredInstPHIRecipe>(UI)) {
+ const VPBlockBase *Incoming = nullptr;
+ // Get the incoming block based on the number of predecessors.
+ if (UserVPBB->getNumPredecessors() == 0) {
+ assert((isa<VPWidenPHIRecipe>(UI) || isa<VPHeaderPHIRecipe>(UI)) &&
+ "Unexpected recipe with 0 predecessors");
+ const VPRegionBlock *UserRegion = UserVPBB->getParent();
+ Incoming = V == UI->getOperand(0)
+ ? UserRegion->getSinglePredecessor()
+ : UserRegion->getExiting();
+ } else if (UserVPBB->getNumPredecessors() == 1) {
+ assert(isa<VPWidenPHIRecipe>(UI) &&
+ "Unexpected recipe with 1 predecessors");
+ Incoming = UserVPBB->getSinglePredecessor();
+ } else {
+ assert(UserVPBB->getNumPredecessors() == 2 &&
+ isa<VPPredInstPHIRecipe>(UI) &&
+ "Unexpected recipe with 2 or more predecessors");
+ Incoming = UserVPBB->getPredecessors()[1];
+ }
+ if (auto *R = dyn_cast<VPRegionBlock>(Incoming))
+ Incoming = R->getExiting();
+ if (Incoming != VPBB && !VPDT.dominates(VPBB, Incoming)) {
+ errs() << "Use before def!\n";
+ return false;
+ }
+ continue;
+ }
// If the user is in the same block, check it comes after R in the
// block.
- if (UI->getParent() == VPBB) {
+ if (UserVPBB == VPBB) {
if (RecipeNumbering[UI] < RecipeNumbering[&R]) {
errs() << "Use before def!\n";
return false;
@@ -222,7 +270,7 @@ bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) {
continue;
}
- if (!VPDT.dominates(VPBB, UI->getParent())) {
+ if (!VPDT.dominates(VPBB, UserVPBB)) {
errs() << "Use before def!\n";
return false;
}
|
@llvm/pr-subscribers-llvm-transforms Author: Florian Hahn (fhahn) ChangesUpdate the verifier to verify dominance for incoming values for phi-like recipes. The defining recipe must dominate the incoming block for the incoming value. There are 4 different cases to consider when retrieving the incoming block:
Full diff: https://github.com/llvm/llvm-project/pull/124838.diff 1 Files Affected:
diff --git a/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp b/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
index 0f151c897d938e..0948452e8d19cf 100644
--- a/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
+++ b/llvm/lib/Transforms/Vectorize/VPlanVerifier.cpp
@@ -175,6 +175,11 @@ bool VPlanVerifier::verifyEVLRecipe(const VPInstruction &EVL) const {
});
}
+/// Return true if \p R is a VPIRInstruction wrapping a phi.
+static bool isVPIRInstructionPhi(const VPRecipeBase &R) {
+ auto *VPIRI = dyn_cast<VPIRInstruction>(&R);
+ return VPIRI && isa<PHINode>(VPIRI->getInstruction());
+}
bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) {
if (!verifyPhiRecipes(VPBB))
return false;
@@ -207,14 +212,57 @@ bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) {
for (const VPUser *U : V->users()) {
auto *UI = cast<VPRecipeBase>(U);
- // TODO: check dominance of incoming values for phis properly.
- if (!UI ||
- isa<VPHeaderPHIRecipe, VPWidenPHIRecipe, VPPredInstPHIRecipe>(UI))
+ const VPBlockBase *UserVPBB = UI->getParent();
+
+ // Verify incoming values of VPIRInstructions wrapping phis. V most
+ // dominate the end of the incoming block. The operand index of the
+ // incoming value matches the predecessor block index of the
+ // corresponding incoming block.
+ if (isVPIRInstructionPhi(*UI)) {
+ for (const auto &[Idx, Op] : enumerate(UI->operands())) {
+ if (V != Op)
+ continue;
+ const VPBlockBase *Incoming = UserVPBB->getPredecessors()[Idx];
+ if (Incoming != VPBB && !VPDT.dominates(VPBB, Incoming)) {
+ errs() << "Use before def!\n";
+ return false;
+ }
+ }
continue;
+ }
+ // Verify incoming value of various phi-like recipes.
+ if (isa<VPWidenPHIRecipe, VPHeaderPHIRecipe, VPPredInstPHIRecipe>(UI)) {
+ const VPBlockBase *Incoming = nullptr;
+ // Get the incoming block based on the number of predecessors.
+ if (UserVPBB->getNumPredecessors() == 0) {
+ assert((isa<VPWidenPHIRecipe>(UI) || isa<VPHeaderPHIRecipe>(UI)) &&
+ "Unexpected recipe with 0 predecessors");
+ const VPRegionBlock *UserRegion = UserVPBB->getParent();
+ Incoming = V == UI->getOperand(0)
+ ? UserRegion->getSinglePredecessor()
+ : UserRegion->getExiting();
+ } else if (UserVPBB->getNumPredecessors() == 1) {
+ assert(isa<VPWidenPHIRecipe>(UI) &&
+ "Unexpected recipe with 1 predecessors");
+ Incoming = UserVPBB->getSinglePredecessor();
+ } else {
+ assert(UserVPBB->getNumPredecessors() == 2 &&
+ isa<VPPredInstPHIRecipe>(UI) &&
+ "Unexpected recipe with 2 or more predecessors");
+ Incoming = UserVPBB->getPredecessors()[1];
+ }
+ if (auto *R = dyn_cast<VPRegionBlock>(Incoming))
+ Incoming = R->getExiting();
+ if (Incoming != VPBB && !VPDT.dominates(VPBB, Incoming)) {
+ errs() << "Use before def!\n";
+ return false;
+ }
+ continue;
+ }
// If the user is in the same block, check it comes after R in the
// block.
- if (UI->getParent() == VPBB) {
+ if (UserVPBB == VPBB) {
if (RecipeNumbering[UI] < RecipeNumbering[&R]) {
errs() << "Use before def!\n";
return false;
@@ -222,7 +270,7 @@ bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) {
continue;
}
- if (!VPDT.dominates(VPBB, UI->getParent())) {
+ if (!VPDT.dominates(VPBB, UserVPBB)) {
errs() << "Use before def!\n";
return false;
}
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for this - looks sensible to me. I just had a couple of comments.
static bool isVPIRInstructionPhi(const VPRecipeBase &R) { | ||
auto *VPIRI = dyn_cast<VPIRInstruction>(&R); | ||
return VPIRI && isa<PHINode>(VPIRI->getInstruction()); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: Add new line after function?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adde a newline, thanks
"Unexpected recipe with 2 or more predecessors"); | ||
Incoming = UserVPBB->getPredecessors()[1]; | ||
} | ||
if (auto *R = dyn_cast<VPRegionBlock>(Incoming)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just out of curiosity, what's the scenario where the incoming block for a VPWidenPHIRecipe, VPHeaderPHIRecipe or VPPredInstPHIRecipe recipe would be the region? Is it the recipes we create in the middle block? If so, could this check potentially fail with uncountable early exits, i.e. the incoming block is effectively the middle.split block? I assume it's fine, but just wondering if there are missing tests for all the scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The only case at the moment is in VPWidenPHIRecipes added in the VPlan-native path. I added an assert; it cannot happen in the inner-loop vectorizer path.
29aa9b0
to
695ef3c
Compare
695ef3c
to
129e8ad
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ping :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good to see this TODO being addressed, adding various comments.
@@ -207,24 +213,69 @@ bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) { | |||
|
|||
for (const VPUser *U : V->users()) { | |||
auto *UI = cast<VPRecipeBase>(U); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
unrelated nit:
auto *UI = cast<VPRecipeBase>(U); | |
auto *UR = cast<VPRecipeBase>(U); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will fix independently.
isa<PHINode>(cast<VPIRInstruction>(UI)->getInstruction()))) | ||
const VPBlockBase *UserVPBB = UI->getParent(); | ||
|
||
// Verify incoming values of VPIRInstructions wrapping phis. V most |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// Verify incoming values of VPIRInstructions wrapping phis. V most | |
// Verify incoming values of VPIRInstructions wrapping phis. V must |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, thanks
|
||
// Verify incoming values of VPIRInstructions wrapping phis. V most | ||
// dominate the end of the incoming block. The operand index of the | ||
// incoming value matches the predecessor block index of the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// incoming value matches the predecessor block index of the | |
// incoming value must match the predecessor block index of the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done thanks!
// incoming value matches the predecessor block index of the | ||
// corresponding incoming block. | ||
if (isVPIRInstructionPhi(*UI)) { | ||
for (const auto &[Idx, Op] : enumerate(UI->operands())) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for (const auto &[Idx, Op] : enumerate(UI->operands())) { | |
for (const auto &[Index, OpAtIndex] : enumerate(UI->operands())) { |
for (const auto &[Idx, Op] : enumerate(UI->operands())) { | ||
if (V != Op) | ||
continue; | ||
const VPBlockBase *Incoming = UserVPBB->getPredecessors()[Idx]; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
const VPBlockBase *Incoming = UserVPBB->getPredecessors()[Idx]; | |
const VPBlockBase *PredAtIndex = UserVPBB->getPredecessors()[Idx]; |
if (isa<VPWidenPHIRecipe, VPHeaderPHIRecipe, VPPredInstPHIRecipe>(UI)) { | ||
const VPBlockBase *Incoming = nullptr; | ||
// Get the incoming block based on the number of predecessors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: better switch over the types of recipes involved, asserting the number of predecessors, than vice-versa? Nearly every recipe seems to be doing its own thing anyhow.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code has been replaced by type-switch and using incoming values/blocks iterator.
"Unexpected recipe with 2 or more predecessors"); | ||
Incoming = UserVPBB->getPredecessors()[1]; | ||
} | ||
if (auto *R = dyn_cast<VPRegionBlock>(Incoming)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
if (auto *R = dyn_cast<VPRegionBlock>(Incoming)) { | |
if (auto *IncomingRegion = dyn_cast<VPRegionBlock>(Incoming)) { |
to avoid confusion with R
which is also being used for the current Recipe.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code is gone in latest version
"VPWidenPHIRecipe"); | ||
Incoming = R->getExiting(); | ||
} | ||
if (Incoming != VPBB && !VPDT.dominates(VPBB, Incoming)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same comment as above.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code is gone in latest version
// dominate the end of the incoming block. The operand index of the | ||
// incoming value matches the predecessor block index of the | ||
// corresponding incoming block. | ||
if (isVPIRInstructionPhi(*UI)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can the case of VPIRInstruction be handled along with all other phi recipes below?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are cases where the block containing a VPIR Phi has multiple predecessors (some cases with early exits), for those we need the proper phi-handling
return false; | ||
} | ||
continue; | ||
} | ||
|
||
// If the user is in the same block, check it comes after R in the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
// If the user is in the same block, check it comes after R in the | |
// If the user is in the same block, check that it comes after R in the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
independent typo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep can fix separately, thanks
Add a VPPhiAccessors class to provide interfaces to access incoming values and blocks, with corresponding iterators. The first user is VPWidenPhiRecipe, with the other phi-like recipes following soon. This will also be used to verify def-use chains where users are phi-like recipes, simplifying llvm#124838.
Add a VPPhiAccessors class to provide interfaces to access incoming values and blocks, with corresponding iterators. The first user is VPWidenPhiRecipe, with the other phi-like recipes following soon. This will also be used to verify def-use chains where users are phi-like recipes, simplifying llvm#124838.
129e8ad
to
bbd3712
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reworked the PR quite a bit and it is now building on a few additional patches:
- Add interface to retrieve incoming blocks & values + iterators, usable for all phi-like recipes: [VPlan] Add new VPIRPhi overlay for VPIRInsts wrapping phi nodes (NFC). #129387
- Add VPIRPhi [VPlan] Add new VPIRPhi overlay for VPIRInsts wrapping phi nodes (NFC). #129387
auto *VPIRI = dyn_cast<VPIRInstruction>(&R); | ||
return VPIRI && isa<PHINode>(VPIRI->getInstruction()); | ||
} | ||
|
||
bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) { | ||
if (!verifyPhiRecipes(VPBB)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure how we could move the new code to verifyPhiRecipes
, as here we verify def before use property looking at the uses of a defined VPValue and when looking at the uses we need to handle phi-like recipes.
@@ -175,6 +175,12 @@ bool VPlanVerifier::verifyEVLRecipe(const VPInstruction &EVL) const { | |||
}); | |||
} | |||
|
|||
/// Return true if \p R is a VPIRInstruction wrapping a phi. | |||
static bool isVPIRInstructionPhi(const VPRecipeBase &R) { | |||
auto *VPIRI = dyn_cast<VPIRInstruction>(&R); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will check separately, thanks!
auto *VPIRI = dyn_cast<VPIRInstruction>(&R); | ||
return VPIRI && isa<PHINode>(VPIRI->getInstruction()); | ||
} | ||
|
||
bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) { | ||
if (!verifyPhiRecipes(VPBB)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It should be complete now, updated, thanks!
@@ -207,24 +213,69 @@ bool VPlanVerifier::verifyVPBasicBlock(const VPBasicBlock *VPBB) { | |||
|
|||
for (const VPUser *U : V->users()) { | |||
auto *UI = cast<VPRecipeBase>(U); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Will fix independently.
isa<PHINode>(cast<VPIRInstruction>(UI)->getInstruction()))) | ||
const VPBlockBase *UserVPBB = UI->getParent(); | ||
|
||
// Verify incoming values of VPIRInstructions wrapping phis. V most |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed, thanks
|
||
// Verify incoming values of VPIRInstructions wrapping phis. V most | ||
// dominate the end of the incoming block. The operand index of the | ||
// incoming value matches the predecessor block index of the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done thanks!
// dominate the end of the incoming block. The operand index of the | ||
// incoming value matches the predecessor block index of the | ||
// corresponding incoming block. | ||
if (isVPIRInstructionPhi(*UI)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are cases where the block containing a VPIR Phi has multiple predecessors (some cases with early exits), for those we need the proper phi-handling
✅ With the latest revision this PR passed the C/C++ code formatter. |
Add a VPPhiAccessors class to provide interfaces to access incoming values and blocks, with corresponding iterators. The first user is VPWidenPhiRecipe, with the other phi-like recipes following soon. This will also be used to verify def-use chains where users are phi-like recipes, simplifying llvm#124838.
…llvm#129388) Add a VPPhiAccessors class to provide interfaces to access incoming values and blocks. The first user is VPWidenPhiRecipe, with the other phi-like recipes following soon. This will also be used to verify def-use chains where users are phi-like recipes, simplifying llvm#124838. PR: llvm#129388
Similarly to VPInstructionWithType and VPIRPhi, add VPPhi as a subclass for VPInstruction. This allows implementing the VPPhiAccessors trait, making available helpers for generic printing of incoming values / blocks and accessors for incoming blocks and values. It will also allow properly verifying def-uses for values used by VPInstructions with PHI opcodes via llvm#124838.
Similarly to VPInstructionWithType and VPIRPhi, add VPPhi as a subclass for VPInstruction. This allows implementing the VPPhiAccessors trait, making available helpers for generic printing of incoming values / blocks and accessors for incoming blocks and values. It will also allow properly verifying def-uses for values used by VPInstructions with PHI opcodes via llvm#124838.
Similarly to VPInstructionWithType and VPIRPhi, add VPPhi as a subclass for VPInstruction. This allows implementing the VPPhiAccessors trait, making available helpers for generic printing of incoming values / blocks and accessors for incoming blocks and values. It will also allow properly verifying def-uses for values used by VPInstructions with PHI opcodes via llvm#124838.
…139151) Similarly to VPInstructionWithType and VPIRPhi, add VPPhi as a subclass for VPInstruction. This allows implementing the VPPhiAccessors trait, making available helpers for generic printing of incoming values / blocks and accessors for incoming blocks and values. It will also allow properly verifying def-uses for values used by VPInstructions with PHI opcodes via #124838. PR: #139151
…des.(NFC) (#139151) Similarly to VPInstructionWithType and VPIRPhi, add VPPhi as a subclass for VPInstruction. This allows implementing the VPPhiAccessors trait, making available helpers for generic printing of incoming values / blocks and accessors for incoming blocks and values. It will also allow properly verifying def-uses for values used by VPInstructions with PHI opcodes via llvm/llvm-project#124838. PR: llvm/llvm-project#139151
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -1135,7 +1135,14 @@ class VPPhiAccessors { | |||
const VPBasicBlock *getIncomingBlock(unsigned Idx) const; | |||
|
|||
/// Returns the number of incoming values, also number of incoming blocks. | |||
unsigned getNumIncoming() const { return getAsRecipe()->getNumOperands(); } | |||
/// Note that at the moment, VPWidenIntOrFpInductionRecipes only have a single |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like this still includes bits of #138472, which looks like it hasn't landed yet?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@@ -1135,7 +1135,14 @@ class VPPhiAccessors { | |||
const VPBasicBlock *getIncomingBlock(unsigned Idx) const; | |||
|
|||
/// Returns the number of incoming values, also number of incoming blocks. | |||
unsigned getNumIncoming() const { return getAsRecipe()->getNumOperands(); } | |||
/// Note that at the moment, VPWidenIntOrFpInductionRecipes only have a single | |||
/// incoming value, its start value. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Better have VPWidenIntOrFpInductionRecipe override a virtual getNumIncoming()?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done thanks
|
||
static inline VPPhiAccessors *castFailed() { return nullptr; } | ||
|
||
static inline VPPhiAccessors *doCastIfPossible(const VPRecipeBase *f) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is doCastIfPossible used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is used by dyn_cast
|
||
using Self = CastInfo<VPPhiAccessors, const VPRecipeBase *>; | ||
|
||
using CastReturnType = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is CastReturnType used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unused in the latest version, removed, thanks!
@@ -3283,6 +3295,48 @@ class VPScalarIVStepsRecipe : public VPRecipeWithIRFlags, | |||
} | |||
}; | |||
|
|||
/// Casting from VPRecipeBase -> VPPhiAccessors is supported for all recipe | |||
/// types implementing VPPhiAccessors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this scaffolding needed for dyn_cast<VPPhiAccessors>(RecipeBase*)
to work, due to its multiple inheritance?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep as there's no link between RecipeBase and VPPhiAccessors. CastIsPossible
is used for isa
/// Support casting from VPRecipeBase -> VPPhiAccessors, by down-casting to the | ||
/// recipe types implementing VPPhiAccessors. | ||
template <> | ||
struct CastInfo<VPPhiAccessors, const VPRecipeBase *> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Where is CastInfo used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is used by cast
/dyn_cast
to perform the actual cast.
const VPBasicBlock *IncVPBB = Phi->getIncomingBlock(Idx); | ||
if (IncVPV != V) | ||
continue; | ||
if (IncVPBB != VPBB && !VPDT.dominates(VPBB, IncVPBB)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dominates(VPBB, IncVPBB)
returns true when VPBB == IncVPBB, so suffice to check if (!VPDT.dominates(VPBB, IncVPBB))
? But is it ok to have VPBB == IncVPBB, given that U is a phi - only if V is a phi too and such dependences between two phi's of the same block are allowed in VPlan, where fixed order dependences are modelled with intervening splices - for VF>1?
In any case good to continue with early continue:
if (IncVPBB != VPBB && !VPDT.dominates(VPBB, IncVPBB)) { | |
if (VPDT.properlyDominates(VPBB, IncVPBB)) | |
continue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated to just dominates
. We could have a loop with a single block, then the incoming value from the latch would be definined in the same block?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, right; a value can be defined in the header and feed a header phi (of the same header block) even when the loop has additional blocks. Can check that in this case it's the 2nd incoming of the phi, i.e., across the backedge.
VPValue *IncVPV = Phi->getIncomingValue(Idx); | ||
const VPBasicBlock *IncVPBB = Phi->getIncomingBlock(Idx); | ||
if (IncVPV != V) | ||
continue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VPValue *IncVPV = Phi->getIncomingValue(Idx); | |
const VPBasicBlock *IncVPBB = Phi->getIncomingBlock(Idx); | |
if (IncVPV != V) | |
continue; | |
VPValue *IncomingV = Phi->getIncomingValue(Idx); | |
if (IncomingV != V) | |
continue; | |
const VPBasicBlock *IncomingVPBB = Phi->getIncomingBlock(Idx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updated, thanks
/// types implementing VPPhiAccessors. | ||
template <> struct CastIsPossible<VPPhiAccessors, const VPRecipeBase *> { | ||
static inline bool isPossible(const VPRecipeBase *f) { | ||
return isa<VPIRPhi, VPHeaderPHIRecipe, VPWidenPHIRecipe, VPPhi>(f); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
VPPredInstPHIRecipe might be another?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, but I left it out for now because more work will be needed as there the operands also don't match the # of predecessors.
} | ||
continue; | ||
} | ||
if (isa<VPPredInstPHIRecipe>(UI)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Anything to check?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yep, added a TODO for now, thanks
return false; | ||
} | ||
continue; | ||
} | ||
|
||
// If the user is in the same block, check it comes after R in the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
independent typo.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks!
@@ -3283,6 +3295,43 @@ class VPScalarIVStepsRecipe : public VPRecipeWithIRFlags, | |||
} | |||
}; | |||
|
|||
/// Casting from VPRecipeBase -> VPPhiAccessors is supported for all recipe | |||
/// types implementing VPPhiAccessors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// types implementing VPPhiAccessors. | |
/// types implementing VPPhiAccessors. CastIsPossible is used by isa. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Adde a note, thanks!
/// Casting from VPRecipeBase -> VPPhiAccessors is supported for all recipe | ||
/// types implementing VPPhiAccessors. | ||
template <> struct CastIsPossible<VPPhiAccessors, const VPRecipeBase *> { | ||
static inline bool isPossible(const VPRecipeBase *f) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
static inline bool isPossible(const VPRecipeBase *f) { | |
static inline bool isPossible(const VPRecipeBase *f) { | |
// TODO: include VPPredInstPHIRecipe too. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added thanks
} | ||
}; | ||
/// Support casting from VPRecipeBase -> VPPhiAccessors, by down-casting to the | ||
/// recipe types implementing VPPhiAccessors. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
/// recipe types implementing VPPhiAccessors. | |
/// recipe types implementing VPPhiAccessors. CastInfo is used by cast and dyn_cast. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added thanks.
}()); | ||
} | ||
|
||
static inline VPPhiAccessors *doCastIfPossible(const VPRecipeBase *f) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
static inline VPPhiAccessors *doCastIfPossible(const VPRecipeBase *f) { | |
/// doCastIfPossible is used by dyn_cast. | |
static inline VPPhiAccessors *doCastIfPossible(const VPRecipeBase *f) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
added thanks
if (IncVPV != V) | ||
continue; | ||
const VPBasicBlock *IncVPBB = Phi->getIncomingBlock(Idx); | ||
if (!VPDT.dominates(VPBB, IncVPBB)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Another early-continue?
if (!VPDT.dominates(VPBB, IncVPBB)) { | |
if (VPDT.dominates(VPBB, IncVPBB)) | |
continue; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated thanks
const VPBasicBlock *IncVPBB = Phi->getIncomingBlock(Idx); | ||
if (IncVPV != V) | ||
continue; | ||
if (IncVPBB != VPBB && !VPDT.dominates(VPBB, IncVPBB)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, right; a value can be defined in the header and feed a header phi (of the same header block) even when the loop has additional blocks. Can check that in this case it's the 2nd incoming of the phi, i.e., across the backedge.
cast<VPInstruction>(UI)->getOpcode() == Instruction::PHI)) | ||
if (auto *Phi = dyn_cast<VPPhiAccessors>(UI)) { | ||
for (unsigned Idx = 0; Idx != Phi->getNumIncoming(); ++Idx) { | ||
VPValue *IncVPV = Phi->getIncomingValue(Idx); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Inc" is ambiguous - e.g., also stands for "Increment". Hence suggested renaming to IncomingV
and IncomingVPBB
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
updated, thanks
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/2/builds/24015 Here is the relevant piece of the build log for the reference
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/51/builds/16196 Here is the relevant piece of the build log for the reference
|
…ecipes. (#124838) Update the verifier to verify dominance for incoming values for phi-like recipes. The defining recipe must dominate the incoming block for the incoming value. Builds on top of llvm/llvm-project#138472 to retrieve incoming values & corresponding blocks for phi-like recipes. PR: llvm/llvm-project#124838
@fhahn , these changes cause a crash for the following tests on the expensive check builders such as llvm-clang-x86_64-expensive-checks-ubuntu:
more details here https://lab.llvm.org/buildbot/#/builders/187/builds/5956
would you fix the problem as soon as it possible or revert these changes |
@vvereschaka should be back to green after b3e7e4b |
Similar to VPWidenIntOrFpInductionRecipe, VPWidenPointerInductionRecipe only has a single incoming value and no backedge value. Currently NFC as there are no problematic uses at the moment.
Update the verifier to verify dominance for incoming values for phi-like recipes. The defining recipe must dominate the incoming block for the incoming value.
Builds on top of #138472 to retrieve incoming values & corresponding blocks for phi-like recipes.