[SDAG] Make Select-with-Identity-Fold More Flexible; NFC #136554
Conversation
|
@llvm/pr-subscribers-backend-risc-v @llvm/pr-subscribers-llvm-selectiondag Author: Marius Kamp (mskamp) ChangesThis change introduces a new overload of the method No functional change is intended because the default (and currently only) implementation restricts the fold to a The rationale of this change is to make more fine grained decisions possible when to revert the InstCombine canonicalization of Full diff: https://github.com/llvm/llvm-project/pull/136554.diff 2 Files Affected:
diff --git a/llvm/include/llvm/CodeGen/TargetLowering.h b/llvm/include/llvm/CodeGen/TargetLowering.h
index 00c36266a069f..ace71bee0ac34 100644
--- a/llvm/include/llvm/CodeGen/TargetLowering.h
+++ b/llvm/include/llvm/CodeGen/TargetLowering.h
@@ -3351,13 +3351,26 @@ class TargetLoweringBase {
}
/// Return true if pulling a binary operation into a select with an identity
- /// constant is profitable. This is the inverse of an IR transform.
+ /// constant is profitable for the given binary operation and type. This is
+ /// the inverse of an IR transform.
/// Example: X + (Cond ? Y : 0) --> Cond ? (X + Y) : X
virtual bool shouldFoldSelectWithIdentityConstant(unsigned BinOpcode,
EVT VT) const {
return false;
}
+ /// Return true if pulling a binary operation into a select with an identity
+ /// constant is profitable for the given binary operation, select operation,
+ /// operand to the binary operation, and select operand that is not the
+ /// identity constant. This is a more fine-grained variant of the previous
+ /// overload that is called only if the previous overload returned true.
+ virtual bool
+ shouldFoldSelectWithIdentityConstant(unsigned BinOpcode,
+ unsigned SelectOpcode, SDValue X,
+ SDValue NonIdConstNode) const {
+ return SelectOpcode == ISD::VSELECT;
+ }
+
/// Return true if it is beneficial to convert a load of a constant to
/// just the constant itself.
/// On some targets it might be more efficient to use a combination of
diff --git a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
index b175e35385ec6..c67614f4aa759 100644
--- a/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
+++ b/llvm/lib/CodeGen/SelectionDAG/DAGCombiner.cpp
@@ -2425,8 +2425,9 @@ static SDValue foldSelectWithIdentityConstant(SDNode *N, SelectionDAG &DAG,
if (ShouldCommuteOperands)
std::swap(N0, N1);
- // TODO: Should this apply to scalar select too?
- if (N1.getOpcode() != ISD::VSELECT || !N1.hasOneUse())
+ unsigned SelOpcode = N1.getOpcode();
+ if ((SelOpcode != ISD::VSELECT && SelOpcode != ISD::SELECT) ||
+ !N1.hasOneUse())
return SDValue();
// We can't hoist all instructions because of immediate UB (not speculatable).
@@ -2439,17 +2440,20 @@ static SDValue foldSelectWithIdentityConstant(SDNode *N, SelectionDAG &DAG,
SDValue Cond = N1.getOperand(0);
SDValue TVal = N1.getOperand(1);
SDValue FVal = N1.getOperand(2);
+ const TargetLowering &TLI = DAG.getTargetLoweringInfo();
// This transform increases uses of N0, so freeze it to be safe.
// binop N0, (vselect Cond, IDC, FVal) --> vselect Cond, N0, (binop N0, FVal)
unsigned OpNo = ShouldCommuteOperands ? 0 : 1;
- if (isNeutralConstant(Opcode, N->getFlags(), TVal, OpNo)) {
+ if (isNeutralConstant(Opcode, N->getFlags(), TVal, OpNo) &&
+ TLI.shouldFoldSelectWithIdentityConstant(Opcode, SelOpcode, N0, FVal)) {
SDValue F0 = DAG.getFreeze(N0);
SDValue NewBO = DAG.getNode(Opcode, SDLoc(N), VT, F0, FVal, N->getFlags());
return DAG.getSelect(SDLoc(N), VT, Cond, F0, NewBO);
}
// binop N0, (vselect Cond, TVal, IDC) --> vselect Cond, (binop N0, TVal), N0
- if (isNeutralConstant(Opcode, N->getFlags(), FVal, OpNo)) {
+ if (isNeutralConstant(Opcode, N->getFlags(), FVal, OpNo) &&
+ TLI.shouldFoldSelectWithIdentityConstant(Opcode, SelOpcode, N0, TVal)) {
SDValue F0 = DAG.getFreeze(N0);
SDValue NewBO = DAG.getNode(Opcode, SDLoc(N), VT, F0, TVal, N->getFlags());
return DAG.getSelect(SDLoc(N), VT, Cond, NewBO, F0);
|
| /// Return true if pulling a binary operation into a select with an identity | ||
| /// constant is profitable for the given binary operation, select operation, | ||
| /// operand to the binary operation, and select operand that is not the | ||
| /// identity constant. This is a more fine-grained variant of the previous | ||
| /// overload that is called only if the previous overload returned true. | ||
| virtual bool | ||
| shouldFoldSelectWithIdentityConstant(unsigned BinOpcode, | ||
| unsigned SelectOpcode, SDValue X, | ||
| SDValue NonIdConstNode) const { | ||
| return SelectOpcode == ISD::VSELECT; | ||
| } |
There was a problem hiding this comment.
Just merge these into one interface?
There was a problem hiding this comment.
I haven't done this because the previous shouldFoldSelectWithIdentityConstant() method is called before calling the foldSelectWithIdentityConstant() method twice (at least for commutative operations). When merging the functions, we would call shouldFoldSelectWithIdentityConstant() four times in the worst case.
But if that is not a problem, I can merge these functions (as this would clearly be a cleaner interface).
There was a problem hiding this comment.
I think you can avoid that with some shuffling around of the code. We're currently calling shouldFoldSelectWithIdentityConstant eagerly, before the other operand parsing logic to see if we can even do this. Could sink this down into the usage function, and handle the attempted commute internal to foldSelectWithIdentityConstant
There was a problem hiding this comment.
But the call to shouldFoldSelectWithIdentityConstant() in foldSelectWithIdentityConstant() depends on the true/false operand of the select, which is only available after handling commutativity of the binary operator. Therefore, we cannot sink the handling of commutativity beyond the call to shouldFoldSelectWithIdentityConstant().
Nevertheless, I've noticed that the mentioned worst case with four calls to shouldFoldSelectWithIdentityConstant() can only happen if we have a select with an identity constant as both the true and false operand. But such operations should have been folded before, so we should have at most two calls to shouldFoldSelectWithIdentityConstant(). Therefore, just merging the methods and keeping only the second call is probably not an issue for performance. I've implemented this suggestion.
This change adds new parameters to the method `shouldFoldSelectWithIdentityConstant()`. The method now takes the opcode of the select node and the non-identity operand of the select node. To gain access to the appropriate arguments, the call of `shouldFoldSelectWithIdentityConstant()` is moved after all other checks have been performed. Moreover, this change adjusts the precondition of the fold so that it would work for `SELECT` nodes in addition to `VSELECT` nodes. No functional change is intended because all implementations of `shouldFoldSelectWithIdentityConstant()` are adjusted such that they restrict the fold to a `VSELECT` node; the same restriction as before. The rationale of this change is to make more fine grained decisions possible when to revert the InstCombine canonicalization of `(select c (binop x y) y)` to `(binop (select c x idc) y)` in the backends.
806fdc7 to
1c2be64
Compare
|
Ping. Any more suggestions for improvement? |
This change adds new parameters to the method
shouldFoldSelectWithIdentityConstant(). The method now takes theopcode of the select node and the non-identity operand of the select
node. To gain access to the appropriate arguments, the call of
shouldFoldSelectWithIdentityConstant()is moved after all other checkshave been performed. Moreover, this change adjusts the precondition of
the fold so that it would work for
SELECTnodes in addition toVSELECTnodes.No functional change is intended because all implementations of
shouldFoldSelectWithIdentityConstant()are adjusted such that theyrestrict the fold to a
VSELECTnode; the same restriction as before.The rationale of this change is to make more fine grained decisions
possible when to revert the InstCombine canonicalization of
(select c (binop x y) y)to(binop (select c x idc) y)in thebackends.