Thanks to visit codestin.com
Credit goes to github.com

Skip to content

IR: Remove uselist for constantdata #137313

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
May 6, 2025

Conversation

arsenm
Copy link
Contributor

@arsenm arsenm commented Apr 25, 2025

This is a resurrected version of the patch attached to this RFC:

https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606

In this adaptation, there are a few differences. In the original patch, the Use's
use list was replaced with an unsigned* to the reference count in the value. This
version leaves them as null and leaves the ref counting only in Value.

Remove use-lists from instances of ConstantData (which are shared
across modules and have no operands).

To continue supporting most of the use-list API, store a ref-count in
place of the use-list; this is for API like Value::use_empty and
Value::hasNUses. Operations that actually need the use-list -- like
Value::use_begin -- will assert.

This change has three benefits:

  1. The compiler output cannot in any way depend on the use-list order
    of instances of ConstantData.

  2. There's no use-list traffic when adding and removing simple
    constants from operand lists (although there is ref-count traffic;
    YMMV).

  3. It's cheaper to serialize use-lists (since we're no longer
    serializing the use-list order of things like i32 0).

The downside is that you can't look at all the users of ConstantData,
but traversals of users of i32 0 are already ill-advised.

Possible follow-ups:

  • Track if an instance of a ConstantVector/ConstantArray/etc. is known
    to have all ConstantData arguments, and drop the use-lists to
    ref-counts in those cases. Callers need to check Value::hasUseList
    before iterating through the use-list.
  • Remove even the ref-counts. I'm not sure they have any benefit
    besides minimizing the scope of this commit, and maintaining the
    counts is not free.

Fixes #58629

Co-authored-by: Duncan P. N. Exon Smith [email protected]

Copy link
Contributor Author

arsenm commented Apr 25, 2025

@llvmbot
Copy link
Member

llvmbot commented Apr 25, 2025

@llvm/pr-subscribers-llvm-transforms
@llvm/pr-subscribers-llvm-analysis
@llvm/pr-subscribers-backend-aarch64

@llvm/pr-subscribers-llvm-ir

Author: Matt Arsenault (arsenm)

Changes

This is a resurrected version of the patch attached to this RFC:

https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606

In this adaptation, there are a few differences. In the original patch, the Use's
use list was replaced with an unsigned* to the reference count in the value. This
version leaves them as null and leaves the ref counting only in Value.

Remove use-lists from instances of ConstantData (which are shared
across modules and have no operands).

To continue supporting most of the use-list API, store a ref-count in
place of the use-list; this is for API like Value::use_empty and
Value::hasNUses. Operations that actually need the use-list -- like
Value::use_begin -- will assert.

This change has three benefits:

  1. The compiler output cannot in any way depend on the use-list order
    of instances of ConstantData.

  2. There's no use-list traffic when adding and removing simple
    constants from operand lists (although there is ref-count traffic;
    YMMV).

  3. It's cheaper to serialize use-lists (since we're no longer
    serializing the use-list order of things like i32 0).

The downside is that you can't look at all the users of ConstantData,
but traversals of users of i32 0 are already ill-advised.

Possible follow-ups:

  • Track if an instance of a ConstantVector/ConstantArray/etc. is known
    to have all ConstantData arguments, and drop the use-lists to
    ref-counts in those cases. Callers need to check Value::hasUseList
    before iterating through the use-list.
  • Remove even the ref-counts. I'm not sure they have any benefit
    besides minimizing the scope of this commit, and maintaining the
    counts is not free.

Fixes #58629


Patch is 30.67 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/137313.diff

26 Files Affected:

  • (modified) llvm/docs/ReleaseNotes.md (+2)
  • (modified) llvm/include/llvm/IR/Constants.h (+2)
  • (modified) llvm/include/llvm/IR/Use.h (+6-17)
  • (modified) llvm/include/llvm/IR/Value.h (+94-24)
  • (modified) llvm/lib/Analysis/TypeMetadataUtils.cpp (+3)
  • (modified) llvm/lib/AsmParser/LLParser.cpp (+2)
  • (modified) llvm/lib/Bitcode/Reader/BitcodeReader.cpp (+4)
  • (modified) llvm/lib/Bitcode/Writer/ValueEnumerator.cpp (+3)
  • (modified) llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp (+1-1)
  • (modified) llvm/lib/CodeGen/CodeGenPrepare.cpp (+3)
  • (modified) llvm/lib/CodeGen/ComplexDeinterleavingPass.cpp (+3)
  • (modified) llvm/lib/IR/AsmWriter.cpp (+7-2)
  • (modified) llvm/lib/IR/Instruction.cpp (+3-1)
  • (modified) llvm/lib/IR/Use.cpp (+6-2)
  • (modified) llvm/lib/IR/Value.cpp (+18-8)
  • (modified) llvm/lib/Target/AArch64/GISel/AArch64RegisterBankInfo.cpp (+1-1)
  • (modified) llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp (+37-29)
  • (modified) llvm/lib/Transforms/InstCombine/InstCombineSimplifyDemanded.cpp (+6-1)
  • (modified) llvm/lib/Transforms/InstCombine/InstructionCombining.cpp (+1-1)
  • (modified) llvm/lib/Transforms/Scalar/Reassociate.cpp (+2-1)
  • (modified) llvm/test/Analysis/MemorySSA/nondeterminism.ll (-1)
  • (added) llvm/test/tools/llvm-diff/uselistorder-issue58629-gv.ll (+14)
  • (modified) llvm/test/tools/llvm-diff/uselistorder-issue58629.ll (+3-2)
  • (modified) llvm/test/tools/llvm-reduce/bitcode-uselistorder.ll (+12-11)
  • (modified) llvm/test/tools/llvm-reduce/uselistorder-invalid-ir-output.ll (+4-2)
  • (modified) llvm/tools/verify-uselistorder/verify-uselistorder.cpp (+9)
diff --git a/llvm/docs/ReleaseNotes.md b/llvm/docs/ReleaseNotes.md
index 6fb206e4df188..411cefe004e16 100644
--- a/llvm/docs/ReleaseNotes.md
+++ b/llvm/docs/ReleaseNotes.md
@@ -56,6 +56,8 @@ Makes programs 10x faster by doing Special New Thing.
 Changes to the LLVM IR
 ----------------------
 
+* It is no longer permitted to inspect the uses of ConstantData
+
 * The `nocapture` attribute has been replaced by `captures(none)`.
 * The constant expression variants of the following instructions have been
   removed:
diff --git a/llvm/include/llvm/IR/Constants.h b/llvm/include/llvm/IR/Constants.h
index a50217078d0ed..7b1dbdece43f7 100644
--- a/llvm/include/llvm/IR/Constants.h
+++ b/llvm/include/llvm/IR/Constants.h
@@ -50,6 +50,8 @@ template <class ConstantClass> struct ConstantAggrKeyType;
 /// These constants have no operands; they represent their data directly.
 /// Since they can be in use by unrelated modules (and are never based on
 /// GlobalValues), it never makes sense to RAUW them.
+///
+/// These do not have use lists. It is illegal to inspect the uses.
 class ConstantData : public Constant {
   constexpr static IntrusiveOperandsAllocMarker AllocMarker{0};
 
diff --git a/llvm/include/llvm/IR/Use.h b/llvm/include/llvm/IR/Use.h
index a86b9c46c1f69..bcd1fd6677497 100644
--- a/llvm/include/llvm/IR/Use.h
+++ b/llvm/include/llvm/IR/Use.h
@@ -23,6 +23,7 @@
 namespace llvm {
 
 template <typename> struct simplify_type;
+class ConstantData;
 class User;
 class Value;
 
@@ -42,10 +43,7 @@ class Use {
 
 private:
   /// Destructor - Only for zap()
-  ~Use() {
-    if (Val)
-      removeFromList();
-  }
+  ~Use();
 
   /// Constructor
   Use(User *Parent) : Parent(Parent) {}
@@ -87,19 +85,10 @@ class Use {
   Use **Prev = nullptr;
   User *Parent = nullptr;
 
-  void addToList(Use **List) {
-    Next = *List;
-    if (Next)
-      Next->Prev = &Next;
-    Prev = List;
-    *Prev = this;
-  }
-
-  void removeFromList() {
-    *Prev = Next;
-    if (Next)
-      Next->Prev = Prev;
-  }
+  inline void addToList(unsigned &Count);
+  inline void addToList(Use *&List);
+  inline void removeFromList(unsigned &Count);
+  inline void removeFromList(Use *&List);
 };
 
 /// Allow clients to treat uses just like values when using
diff --git a/llvm/include/llvm/IR/Value.h b/llvm/include/llvm/IR/Value.h
index bf1de7eef9932..180b6238eda6c 100644
--- a/llvm/include/llvm/IR/Value.h
+++ b/llvm/include/llvm/IR/Value.h
@@ -116,7 +116,10 @@ class Value {
 
 private:
   Type *VTy;
-  Use *UseList;
+  union {
+    Use *List = nullptr;
+    unsigned Count;
+  } Uses;
 
   friend class ValueAsMetadata; // Allow access to IsUsedByMD.
   friend class ValueHandleBase; // Allow access to HasValueHandle.
@@ -339,21 +342,28 @@ class Value {
 #endif
   }
 
+  /// Check if this Value has a use-list.
+  bool hasUseList() const { return !isa<ConstantData>(this); }
+
   bool use_empty() const {
     assertModuleIsMaterialized();
-    return UseList == nullptr;
+    return hasUseList() ? Uses.List == nullptr : Uses.Count == 0;
   }
 
   bool materialized_use_empty() const {
-    return UseList == nullptr;
+    return hasUseList() ? Uses.List == nullptr : !Uses.Count;
   }
 
   using use_iterator = use_iterator_impl<Use>;
   using const_use_iterator = use_iterator_impl<const Use>;
 
-  use_iterator materialized_use_begin() { return use_iterator(UseList); }
+  use_iterator materialized_use_begin() {
+    assert(hasUseList());
+    return use_iterator(Uses.List);
+  }
   const_use_iterator materialized_use_begin() const {
-    return const_use_iterator(UseList);
+    assert(hasUseList());
+    return const_use_iterator(Uses.List);
   }
   use_iterator use_begin() {
     assertModuleIsMaterialized();
@@ -380,17 +390,18 @@ class Value {
     return materialized_uses();
   }
 
-  bool user_empty() const {
-    assertModuleIsMaterialized();
-    return UseList == nullptr;
-  }
+  bool user_empty() const { return use_empty(); }
 
   using user_iterator = user_iterator_impl<User>;
   using const_user_iterator = user_iterator_impl<const User>;
 
-  user_iterator materialized_user_begin() { return user_iterator(UseList); }
+  user_iterator materialized_user_begin() {
+    assert(hasUseList());
+    return user_iterator(Uses.List);
+  }
   const_user_iterator materialized_user_begin() const {
-    return const_user_iterator(UseList);
+    assert(hasUseList());
+    return const_user_iterator(Uses.List);
   }
   user_iterator user_begin() {
     assertModuleIsMaterialized();
@@ -429,7 +440,11 @@ class Value {
   ///
   /// This is specialized because it is a common request and does not require
   /// traversing the whole use list.
-  bool hasOneUse() const { return hasSingleElement(uses()); }
+  bool hasOneUse() const {
+    if (!hasUseList())
+      return Uses.Count == 1;
+    return hasSingleElement(uses());
+  }
 
   /// Return true if this Value has exactly N uses.
   bool hasNUses(unsigned N) const;
@@ -491,6 +506,8 @@ class Value {
   static void dropDroppableUse(Use &U);
 
   /// Check if this value is used in the specified basic block.
+  ///
+  /// Not supported for ConstantData.
   bool isUsedInBasicBlock(const BasicBlock *BB) const;
 
   /// This method computes the number of uses of this Value.
@@ -500,7 +517,19 @@ class Value {
   unsigned getNumUses() const;
 
   /// This method should only be used by the Use class.
-  void addUse(Use &U) { U.addToList(&UseList); }
+  void addUse(Use &U) {
+    if (hasUseList())
+      U.addToList(Uses.List);
+    else
+      U.addToList(Uses.Count);
+  }
+
+  void removeUse(Use &U) {
+    if (hasUseList())
+      U.removeFromList(Uses.List);
+    else
+      U.removeFromList(Uses.Count);
+  }
 
   /// Concrete subclass of this.
   ///
@@ -841,7 +870,8 @@ class Value {
   ///
   /// \return the first element in the list.
   ///
-  /// \note Completely ignores \a Use::Prev (doesn't read, doesn't update).
+  /// \note Completely ignores \a Use::PrevOrCount (doesn't read, doesn't
+  /// update).
   template <class Compare>
   static Use *mergeUseLists(Use *L, Use *R, Compare Cmp) {
     Use *Merged;
@@ -887,10 +917,50 @@ inline raw_ostream &operator<<(raw_ostream &OS, const Value &V) {
   return OS;
 }
 
+inline Use::~Use() {
+  if (Val)
+    Val->removeUse(*this);
+}
+
+void Use::addToList(unsigned &Count) {
+  assert(isa<ConstantData>(Val) && "Only ConstantData is ref-counted");
+  ++Count;
+
+  // We don't have a uselist - clear the remnant if we are replacing a
+  // non-constant value.
+  Prev = nullptr;
+  Next = nullptr;
+}
+
+void Use::addToList(Use *&List) {
+  assert(!isa<ConstantData>(Val) && "ConstantData has no use-list");
+
+  Next = List;
+  if (Next)
+    Next->Prev = &Next;
+  Prev = &List;
+  List = this;
+}
+
+void Use::removeFromList(unsigned &Count) {
+  assert(isa<ConstantData>(Val));
+  assert(Count > 0 && "reference count underflow");
+  assert(!Prev && !Next && "should not have uselist remnant");
+  --Count;
+}
+
+void Use::removeFromList(Use *&List) {
+  *Prev = Next;
+  if (Next)
+    Next->Prev = Prev;
+}
+
 void Use::set(Value *V) {
-  if (Val) removeFromList();
+  if (Val)
+    Val->removeUse(*this);
   Val = V;
-  if (V) V->addUse(*this);
+  if (V)
+    V->addUse(*this);
 }
 
 Value *Use::operator=(Value *RHS) {
@@ -904,7 +974,7 @@ const Use &Use::operator=(const Use &RHS) {
 }
 
 template <class Compare> void Value::sortUseList(Compare Cmp) {
-  if (!UseList || !UseList->Next)
+  if (!hasUseList() || !Uses.List || !Uses.List->Next)
     // No need to sort 0 or 1 uses.
     return;
 
@@ -917,10 +987,10 @@ template <class Compare> void Value::sortUseList(Compare Cmp) {
   Use *Slots[MaxSlots];
 
   // Collect the first use, turning it into a single-item list.
-  Use *Next = UseList->Next;
-  UseList->Next = nullptr;
+  Use *Next = Uses.List->Next;
+  Uses.List->Next = nullptr;
   unsigned NumSlots = 1;
-  Slots[0] = UseList;
+  Slots[0] = Uses.List;
 
   // Collect all but the last use.
   while (Next->Next) {
@@ -956,15 +1026,15 @@ template <class Compare> void Value::sortUseList(Compare Cmp) {
   // Merge all the lists together.
   assert(Next && "Expected one more Use");
   assert(!Next->Next && "Expected only one Use");
-  UseList = Next;
+  Uses.List = Next;
   for (unsigned I = 0; I < NumSlots; ++I)
     if (Slots[I])
-      // Since the uses in Slots[I] originally preceded those in UseList, send
+      // Since the uses in Slots[I] originally preceded those in Uses.List, send
       // Slots[I] in as the left parameter to maintain a stable sort.
-      UseList = mergeUseLists(Slots[I], UseList, Cmp);
+      Uses.List = mergeUseLists(Slots[I], Uses.List, Cmp);
 
   // Fix the Prev pointers.
-  for (Use *I = UseList, **Prev = &UseList; I; I = I->Next) {
+  for (Use *I = Uses.List, **Prev = &Uses.List; I; I = I->Next) {
     I->Prev = Prev;
     Prev = &I->Next;
   }
diff --git a/llvm/lib/Analysis/TypeMetadataUtils.cpp b/llvm/lib/Analysis/TypeMetadataUtils.cpp
index 9ec0785eb5034..8099fbc3daeda 100644
--- a/llvm/lib/Analysis/TypeMetadataUtils.cpp
+++ b/llvm/lib/Analysis/TypeMetadataUtils.cpp
@@ -54,6 +54,9 @@ findCallsAtConstantOffset(SmallVectorImpl<DevirtCallSite> &DevirtCalls,
 static void findLoadCallsAtConstantOffset(
     const Module *M, SmallVectorImpl<DevirtCallSite> &DevirtCalls, Value *VPtr,
     int64_t Offset, const CallInst *CI, DominatorTree &DT) {
+  if (!VPtr->hasUseList())
+    return;
+
   for (const Use &U : VPtr->uses()) {
     Value *User = U.getUser();
     if (isa<BitCastInst>(User)) {
diff --git a/llvm/lib/AsmParser/LLParser.cpp b/llvm/lib/AsmParser/LLParser.cpp
index af0422f09bc4f..4d6d224cacbf5 100644
--- a/llvm/lib/AsmParser/LLParser.cpp
+++ b/llvm/lib/AsmParser/LLParser.cpp
@@ -8857,6 +8857,8 @@ bool LLParser::parseMDNodeVector(SmallVectorImpl<Metadata *> &Elts) {
 //===----------------------------------------------------------------------===//
 bool LLParser::sortUseListOrder(Value *V, ArrayRef<unsigned> Indexes,
                                 SMLoc Loc) {
+  if (!V->hasUseList())
+    return false;
   if (V->use_empty())
     return error(Loc, "value has no uses");
 
diff --git a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
index b152adfbd93d8..dfda8beecce57 100644
--- a/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
+++ b/llvm/lib/Bitcode/Reader/BitcodeReader.cpp
@@ -3856,6 +3856,10 @@ Error BitcodeReader::parseUseLists() {
         V = FunctionBBs[ID];
       } else
         V = ValueList[ID];
+
+      if (isa<ConstantData>(V))
+        break;
+
       unsigned NumUses = 0;
       SmallDenseMap<const Use *, unsigned, 16> Order;
       for (const Use &U : V->materialized_uses()) {
diff --git a/llvm/lib/Bitcode/Writer/ValueEnumerator.cpp b/llvm/lib/Bitcode/Writer/ValueEnumerator.cpp
index 9f735f77d29dc..0c81a99f2235b 100644
--- a/llvm/lib/Bitcode/Writer/ValueEnumerator.cpp
+++ b/llvm/lib/Bitcode/Writer/ValueEnumerator.cpp
@@ -230,6 +230,9 @@ static void predictValueUseListOrderImpl(const Value *V, const Function *F,
 
 static void predictValueUseListOrder(const Value *V, const Function *F,
                                      OrderMap &OM, UseListOrderStack &Stack) {
+  if (isa<ConstantData>(V))
+    return;
+
   auto &IDPair = OM[V];
   assert(IDPair.first && "Unmapped value");
   if (IDPair.second)
diff --git a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
index bdcd54a135da9..4b0e5893b34e0 100644
--- a/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
+++ b/llvm/lib/CodeGen/AsmPrinter/AsmPrinter.cpp
@@ -4002,7 +4002,7 @@ static void emitGlobalConstantImpl(const DataLayout &DL, const Constant *CV,
   // Globals with sub-elements such as combinations of arrays and structs
   // are handled recursively by emitGlobalConstantImpl. Keep track of the
   // constant symbol base and the current position with BaseCV and Offset.
-  if (!BaseCV && CV->hasOneUse())
+  if (!isa<ConstantData>(CV) && !BaseCV && CV->hasOneUse())
     BaseCV = dyn_cast<Constant>(CV->user_back());
 
   if (isa<ConstantAggregateZero>(CV)) {
diff --git a/llvm/lib/CodeGen/CodeGenPrepare.cpp b/llvm/lib/CodeGen/CodeGenPrepare.cpp
index e8dc7752b23c0..2cb73c3920716 100644
--- a/llvm/lib/CodeGen/CodeGenPrepare.cpp
+++ b/llvm/lib/CodeGen/CodeGenPrepare.cpp
@@ -8591,6 +8591,9 @@ static bool optimizeBranch(BranchInst *Branch, const TargetLowering &TLI,
     return false;
 
   Value *X = Cmp->getOperand(0);
+  if (!X->hasUseList())
+    return false;
+
   APInt CmpC = cast<ConstantInt>(Cmp->getOperand(1))->getValue();
 
   for (auto *U : X->users()) {
diff --git a/llvm/lib/CodeGen/ComplexDeinterleavingPass.cpp b/llvm/lib/CodeGen/ComplexDeinterleavingPass.cpp
index 790edf12b46be..4c5b906ac1d00 100644
--- a/llvm/lib/CodeGen/ComplexDeinterleavingPass.cpp
+++ b/llvm/lib/CodeGen/ComplexDeinterleavingPass.cpp
@@ -1034,6 +1034,9 @@ ComplexDeinterleavingGraph::identifyPartialReduction(Value *R, Value *I) {
   if (!isa<VectorType>(R->getType()) || !isa<VectorType>(I->getType()))
     return nullptr;
 
+  if (!R->hasUseList() || !I->hasUseList())
+    return nullptr;
+
   auto CommonUser =
       findCommonBetweenCollections<Value *>(R->users(), I->users());
   if (!CommonUser)
diff --git a/llvm/lib/IR/AsmWriter.cpp b/llvm/lib/IR/AsmWriter.cpp
index ac8aa0d35ea30..fedad23066084 100644
--- a/llvm/lib/IR/AsmWriter.cpp
+++ b/llvm/lib/IR/AsmWriter.cpp
@@ -125,11 +125,15 @@ static void orderValue(const Value *V, OrderMap &OM) {
   if (OM.lookup(V))
     return;
 
-  if (const Constant *C = dyn_cast<Constant>(V))
+  if (const Constant *C = dyn_cast<Constant>(V)) {
+    if (isa<ConstantData>(C))
+      return;
+
     if (C->getNumOperands() && !isa<GlobalValue>(C))
       for (const Value *Op : C->operands())
         if (!isa<BasicBlock>(Op) && !isa<GlobalValue>(Op))
           orderValue(Op, OM);
+  }
 
   // Note: we cannot cache this lookup above, since inserting into the map
   // changes the map's size, and thus affects the other IDs.
@@ -275,7 +279,8 @@ static UseListOrderMap predictUseListOrder(const Module *M) {
   UseListOrderMap ULOM;
   for (const auto &Pair : OM) {
     const Value *V = Pair.first;
-    if (V->use_empty() || std::next(V->use_begin()) == V->use_end())
+    if (!V->hasUseList() || V->use_empty() ||
+        std::next(V->use_begin()) == V->use_end())
       continue;
 
     std::vector<unsigned> Shuffle =
diff --git a/llvm/lib/IR/Instruction.cpp b/llvm/lib/IR/Instruction.cpp
index 4eadecb54feb5..19dd68b3a011d 100644
--- a/llvm/lib/IR/Instruction.cpp
+++ b/llvm/lib/IR/Instruction.cpp
@@ -373,7 +373,9 @@ std::optional<BasicBlock::iterator> Instruction::getInsertionPointAfterDef() {
 }
 
 bool Instruction::isOnlyUserOfAnyOperand() {
-  return any_of(operands(), [](Value *V) { return V->hasOneUser(); });
+  return any_of(operands(), [](const Value *V) {
+    return V->hasUseList() && V->hasOneUser();
+  });
 }
 
 void Instruction::setHasNoUnsignedWrap(bool b) {
diff --git a/llvm/lib/IR/Use.cpp b/llvm/lib/IR/Use.cpp
index 99a89386d75f9..67882ba0144b4 100644
--- a/llvm/lib/IR/Use.cpp
+++ b/llvm/lib/IR/Use.cpp
@@ -19,11 +19,15 @@ void Use::swap(Use &RHS) {
   std::swap(Next, RHS.Next);
   std::swap(Prev, RHS.Prev);
 
-  *Prev = this;
+  if (Prev)
+    *Prev = this;
+
   if (Next)
     Next->Prev = &Next;
 
-  *RHS.Prev = &RHS;
+  if (RHS.Prev)
+    *RHS.Prev = &RHS;
+
   if (RHS.Next)
     RHS.Next->Prev = &RHS.Next;
 }
diff --git a/llvm/lib/IR/Value.cpp b/llvm/lib/IR/Value.cpp
index aa97b70f21aeb..74a96051f33af 100644
--- a/llvm/lib/IR/Value.cpp
+++ b/llvm/lib/IR/Value.cpp
@@ -53,7 +53,7 @@ static inline Type *checkType(Type *Ty) {
 Value::Value(Type *ty, unsigned scid)
     : SubclassID(scid), HasValueHandle(0), SubclassOptionalData(0),
       SubclassData(0), NumUserOperands(0), IsUsedByMD(false), HasName(false),
-      HasMetadata(false), VTy(checkType(ty)), UseList(nullptr) {
+      HasMetadata(false), VTy(checkType(ty)) {
   static_assert(ConstantFirstVal == 0, "!(SubclassID < ConstantFirstVal)");
   // FIXME: Why isn't this in the subclass gunk??
   // Note, we cannot call isa<CallInst> before the CallInst has been
@@ -148,10 +148,14 @@ void Value::destroyValueName() {
 }
 
 bool Value::hasNUses(unsigned N) const {
+  if (!hasUseList())
+    return Uses.Count == N;
   return hasNItems(use_begin(), use_end(), N);
 }
 
 bool Value::hasNUsesOrMore(unsigned N) const {
+  if (!hasUseList())
+    return Uses.Count >= N;
   return hasNItemsOrMore(use_begin(), use_end(), N);
 }
 
@@ -232,6 +236,8 @@ void Value::dropDroppableUse(Use &U) {
 }
 
 bool Value::isUsedInBasicBlock(const BasicBlock *BB) const {
+  assert(hasUseList() && "ConstantData has no use-list");
+
   // This can be computed either by scanning the instructions in BB, or by
   // scanning the use list of this Value. Both lists can be very long, but
   // usually one is quite short.
@@ -253,6 +259,9 @@ bool Value::isUsedInBasicBlock(const BasicBlock *BB) const {
 }
 
 unsigned Value::getNumUses() const {
+  if (!hasUseList())
+    return Uses.Count;
+
   return (unsigned)std::distance(use_begin(), use_end());
 }
 
@@ -499,6 +508,7 @@ static bool contains(Value *Expr, Value *V) {
 #endif // NDEBUG
 
 void Value::doRAUW(Value *New, ReplaceMetadataUses ReplaceMetaUses) {
+  assert(hasUseList() && "Cannot replace constant data");
   assert(New && "Value::replaceAllUsesWith(<null>) is invalid!");
   assert(!contains(New, this) &&
          "this->replaceAllUsesWith(expr(this)) is NOT valid!");
@@ -512,7 +522,7 @@ void Value::doRAUW(Value *New, ReplaceMetadataUses ReplaceMetaUses) {
     ValueAsMetadata::handleRAUW(this, New);
 
   while (!materialized_use_empty()) {
-    Use &U = *UseList;
+    Use &U = *Uses.List;
     // Must handle Constants specially, we cannot call replaceUsesOfWith on a
     // constant because they are uniqued.
     if (auto *C = dyn_cast<Constant>(U.getUser())) {
@@ -844,7 +854,7 @@ bool Value::canBeFreed() const {
   // which is why we need the explicit opt in on a per collector basis.
   if (!F->hasGC())
     return true;
-  
+
   const auto &GCName = F->getGC();
   if (GCName == "statepoint-example") {
     auto *PT = cast<PointerType>(this->getType());
@@ -1092,12 +1102,12 @@ const Value *Value::DoPHITranslation(const BasicBlock *CurBB,
 LLVMContext &Value::getContext() const { return VTy->getContext(); }
 
 void Value::reverseUseList() {
-  if (!UseList || !UseList->Next)
+  if (!Uses.List || !Uses.List->Next || !hasUseList())
     // No need to reverse 0 or 1 uses.
     return;
 
-  Use *Head = UseList;
-  Use *Current = UseList->Next;
+  Use *Head = Uses.List;
+  Use *Current = Uses.List->Next;
   Head->Next = nullptr;
   while (Current) {
     Use *Next = Current->Next;
@@ -1106,8 +1116,8 @@ void Value::reverseUseList() {
     Head = Current;
     Current = Next;
   }
-  UseList = Head;
-  Head->Prev = &UseList;
+  Uses.List = Head;
+  Head->Prev = &Uses.List;
 }
 
 bool Value::isSwiftError() const {
diff --git a/llvm/lib/Target/AArch64/GISel/AArch64RegisterBankInfo.cpp b/llvm/lib/Target/AArch64/GISel/AArch64RegisterBankInfo.cpp
index 8d83fef265e6f..6bd3fd182485d 100644
--- a/llvm/lib/Target/AArch64/GISel/AArch64RegisterBankInfo.cpp
+++ b/llvm/lib/Target/AArch64/GISel/AArch64RegisterBankInfo.cpp
@@ -633,7 +633,7 @@ bool AArch64RegisterBankInfo::isLoadFromFPType(const MachineInstr &MI) const {
     // Look at the first element of the array to determine its type
     if (isa<ArrayType>(EltTy))
       EltTy = EltTy->getArrayElementType();
-  } else {
+  } else if (!isa<Constant>(LdVal)) {
     // FIXME: grubbing around uses is pretty ugly, but with no more
     // `getPointerElementType` there's not much else we can do.
     for (const auto *LdUser : LdVal->users()) {
diff --git a/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp b/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp
index 6205dfedb79fb..c7fdd7dd0a291 100644
--- a/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp
+++ b/llvm/lib/Target/SPIRV/SPIRVEmitIntrinsics.cpp
@@ -1104,7 +1104,7 @@ void SPIRVEmitIntrinsics::deduceOperandElementType(
   IRBuilder<> B(Ctx);
   for (auto &OpIt : Ops) {
     Value *Op = OpIt.first;
-    if (Op->use_empty())
+    if (!Op->hasUseList() || Op->use_empty())
       continue;
     if (AskOps && !AskOps->contains(Op))
       continue;
@@ -1470,34 +1470,36 @@ void SPIRVEmitIntrinsics::replacePointerOperandWithPtrCast(
   // Do not emit new spv_ptrcast if equivalent one already exists or when
   // spv_assign_ptr_type already targets t...
[truncated]

Copy link
Contributor

@nikic nikic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there are some polly assertion failures.

@arsenm arsenm force-pushed the users/arsenm/ir/remove-uselist-for-constantdata branch from 9c95bd9 to 622d957 Compare April 25, 2025 14:18
@@ -1,7 +1,6 @@
; RUN: opt -passes=simplifycfg -S --preserve-ll-uselistorder %s | FileCheck %s
; REQUIRES: x86-registered-target
; CHECK-LABEL: @n
; CHECK: uselistorder i16 0, { 3, 2, 4, 1, 5, 0, 6 }

; Note: test was added in an effort to ensure determinism when updating memoryssa. See PR42574.
; If the uselistorder check becomes no longer relevant, the test can be disabled or removed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment indicates the test should just be deleted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From the context of the commit which added it, I'm not sure what the direct connection is to the uselistorder of a constant. If it's testing nondeterministic map iteration, it would still be covering that despite the uselistorder?

@arsenm arsenm force-pushed the users/arsenm/ir/remove-uselist-for-constantdata branch from 622d957 to e445051 Compare May 6, 2025 06:44
Copy link
Contributor

@nikic nikic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -395,6 +398,9 @@ static void verifyUseListOrder(const Module &M) {

static void shuffleValueUseLists(Value *V, std::minstd_rand0 &Gen,
DenseSet<Value *> &Seen) {
if (isa<ConstantData>(V))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (isa<ConstantData>(V))
if (!V->hasUseList())

@@ -437,6 +443,9 @@ static void shuffleValueUseLists(Value *V, std::minstd_rand0 &Gen,
}

static void reverseValueUseLists(Value *V, DenseSet<Value *> &Seen) {
if (isa<ConstantData>(V))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (isa<ConstantData>(V))
if (V->hasUseList())

@arsenm arsenm force-pushed the users/arsenm/ir/remove-uselist-for-constantdata branch from e445051 to 5101291 Compare May 6, 2025 11:45
Copy link
Contributor Author

arsenm commented May 6, 2025

Merge activity

  • May 6, 11:13 AM EDT: A user started a stack merge that includes this pull request via Graphite.
  • May 6, 11:15 AM EDT: Graphite rebased this pull request as part of a merge.
  • May 6, 11:18 AM EDT: Graphite rebased this pull request as part of a merge.
  • May 6, 11:20 AM EDT: @arsenm merged this pull request with Graphite.

@arsenm arsenm force-pushed the users/arsenm/ir/remove-uselist-for-constantdata branch from 5101291 to fc8ab4c Compare May 6, 2025 15:15
arsenm added 3 commits May 6, 2025 15:17
This is a resurrected version of the patch attached to this RFC:

https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606

In this adaptation, there are a few differences. In the original patch, the Use's
use list was replaced with an unsigned* to the reference count in the value. This
version leaves them as null and leaves the ref counting only in Value.

Remove use-lists from instances of ConstantData (which are shared
across modules and have no operands).

To continue supporting most of the use-list API, store a ref-count in
place of the use-list; this is for API like Value::use_empty and
Value::hasNUses.  Operations that actually need the use-list -- like
Value::use_begin -- will assert.

This change has three benefits:

 1. The compiler output cannot in any way depend on the use-list order
    of instances of ConstantData.

 2. There's no use-list traffic when adding and removing simple
    constants from operand lists (although there is ref-count traffic;
    YMMV).

 3. It's cheaper to serialize use-lists (since we're no longer
    serializing the use-list order of things like i32 0).

The downside is that you can't look at all the users of ConstantData,
but traversals of users of i32 0 are already ill-advised.

Possible follow-ups:
  - Track if an instance of a ConstantVector/ConstantArray/etc. is known
    to have all ConstantData arguments, and drop the use-lists to
    ref-counts in those cases.  Callers need to check Value::hasUseList
    before iterating through the use-list.
  - Remove even the ref-counts.  I'm not sure they have any benefit
    besides minimizing the scope of this commit, and maintaining the
    counts is not free.

Fixes #58629
@arsenm arsenm force-pushed the users/arsenm/ir/remove-uselist-for-constantdata branch from fc8ab4c to 4c5b76e Compare May 6, 2025 15:17
@arsenm arsenm merged commit 87f312a into main May 6, 2025
8 of 11 checks passed
@arsenm arsenm deleted the users/arsenm/ir/remove-uselist-for-constantdata branch May 6, 2025 15:20
@kstoimenov
Copy link
Contributor

@arsenm looks like either this PR or pr/137314 broke sanitizer bots. Could you please take a look: https://lab.llvm.org/buildbot/#/builders/85/builds/8454

kstoimenov added a commit that referenced this pull request May 7, 2025
@mstorsjo
Copy link
Member

mstorsjo commented May 7, 2025

I also bisected a breakage down to this commit. It can be triggered with this reduced input:

struct QTransform {
  double m_matrix[3];
};
struct QPaintEngineState {
  int dirtyFlags;
};
struct QPaintEngine {
  QPaintEngineState *state;
};
struct QPainterState : QPaintEngineState {
  virtual ~QPainterState();
  QTransform matrix;
};
struct QPaintEngineEx : QPaintEngine {
  void stroke();
  QPainterState *state() {
    return static_cast<QPainterState *>(QPaintEngine::state);
  }
};
void QPaintEngineEx::stroke() {
  QPainterState __trans_tmp_1;
  QTransform xform = state()->matrix;
  state()->matrix = QTransform();
  __trans_tmp_1.matrix = xform;
}

Compiled e.g. like this:

$ clang -target x86_64-linux-gnu -c repro.cpp -O2
clang: ../include/llvm/IR/Value.h:360: llvm::Value::const_use_iterator llvm::Value::materialized_use_begin() const: Assertion `hasUseList()' failed.

@dyung
Copy link
Collaborator

dyung commented May 7, 2025

We also saw the same assertion failure in our internal testing and I was able to get it with the following code:

struct a typedef *b;
#define c(d, col) y[e::f]
struct e {
  enum { f, g };
  float y[g];
};
#define h(d, col) i[d]
enum { g };
float i[g];
struct j {
  j(e k) {
    h(2, ) = k.c(, );
    h(3, ) = 0.0f;
  }
};
struct l {
  void m() { j mat(n); }
  e n;
};
struct o {
  int p;
};
struct q {
  l r;
  l s() { return r; }
};
struct a : q, o {};
o *t;
void u() {
  a v = *b(t), w = v;
  l x = w.s();
  x.m();
}

Compile it with optimizations enabled and -fno-delete-null-pointer-checks to trigger the assertion:

$ ~/src/upstream/87f312aad6ed-linux/bin/clang -c -fno-delete-null-pointer-checks -O2 repro.cpp
clang: /home/dyung/src/upstream/llvm_clean_git/llvm/include/llvm/IR/Value.h:365: llvm::Value::const_use_iterator llvm::Value::materialized_use_begin() const: Assertion `hasUseList()' failed.

@kstoimenov
Copy link
Contributor

@arsenm I ended up reverting it. Please make sure to resubmit.

GeorgeARM pushed a commit to GeorgeARM/llvm-project that referenced this pull request May 7, 2025
This is a resurrected version of the patch attached to this RFC:

https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606

In this adaptation, there are a few differences. In the original patch, the Use's
use list was replaced with an unsigned* to the reference count in the value. This
version leaves them as null and leaves the ref counting only in Value.

Remove use-lists from instances of ConstantData (which are shared
across modules and have no operands).

To continue supporting most of the use-list API, store a ref-count in
place of the use-list; this is for API like Value::use_empty and
Value::hasNUses.  Operations that actually need the use-list -- like
Value::use_begin -- will assert.

This change has three benefits:

 1. The compiler output cannot in any way depend on the use-list order
    of instances of ConstantData.

 2. There's no use-list traffic when adding and removing simple
    constants from operand lists (although there is ref-count traffic;
    YMMV).

 3. It's cheaper to serialize use-lists (since we're no longer
    serializing the use-list order of things like i32 0).

The downside is that you can't look at all the users of ConstantData,
but traversals of users of i32 0 are already ill-advised.

Possible follow-ups:
  - Track if an instance of a ConstantVector/ConstantArray/etc. is known
    to have all ConstantData arguments, and drop the use-lists to
    ref-counts in those cases.  Callers need to check Value::hasUseList
    before iterating through the use-list.
  - Remove even the ref-counts.  I'm not sure they have any benefit
    besides minimizing the scope of this commit, and maintaining the
    counts is not free.

Fixes llvm#58629

Co-authored-by: Duncan P. N. Exon Smith <[email protected]>
GeorgeARM pushed a commit to GeorgeARM/llvm-project that referenced this pull request May 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

uselistorder is broken for tools which load multiple files
7 participants