uselistorder is broken for tools which load multiple files #58629

arsenm · 2022-10-26T14:19:55Z

Tools that load multiple files, such as llvm-diff, cannot handle 2 modules using uselistorder. On the parse of the second module, it sees the uses of constants in the first module, and errors when the indexes mismatch. The error is hard for text IR, and silently ignored for bitcode.

The error is failing on this check:

llvm-project/llvm/lib/AsmParser/LLParser.cpp

Line 7845 in 7bc7f2d

if (Order.size() != Indexes.size() || NumUses > Indexes.size())

The equivalent in the bitcode reader simply drops the uselistorder:

llvm-project/llvm/lib/Bitcode/Reader/BitcodeReader.cpp

Line 3485 in 7bc7f2d

if (Order.size() != Record.size() || NumUses > Record.size())

A simple test which tries to run llvm-diff on the same input shows the issue:

uselistorder.ll:10:1: error: wrong number of indexes, expected 4
uselistorder ptr addrspace(4) null, { 1, 0 }

; RUN: llvm-diff %s %s

define void @func() {
entry:
  %gep0 = getelementptr inbounds i8, ptr addrspace(4) null, i64 12
  %gep1 = getelementptr i8, ptr addrspace(4) null, i64 4
  ret void
}

uselistorder ptr addrspace(4) null, { 1, 0 }

I'm not sure what the right solution is. We could parse these into their own contexts, but llvm-diff needs the two modules to share the context to compare. Alternatively, the bitcode parsing could check if the users belong to the same parent module when counting and sorting

The text was updated successfully, but these errors were encountered:

arsenm · 2022-11-11T22:51:09Z

Pushed xfailed test as 7c0362b

DaniilSuchkov · 2023-09-18T16:16:22Z

In case anybody else encounters this issue while trying to use llvm-reduce: you can work around this issue by using --write-tmp-files-as-bitcode.

dexonsmith · 2023-11-30T20:30:39Z

Tools that load multiple files, such as llvm-diff, cannot handle 2 modules using uselistorder. On the parse of the second module, it sees the uses of constants in the first module, and errors when the indexes mismatch. The error is hard for text IR, and silently ignored for bitcode.

If there's a way to detect that the LLVMContext isn't "fresh" (maybe by passing in a flag), then it would be reasonable to silently ignore this error for pure constants. There simply isn't a way to replicate the use-list-order in that situation. (But for a fresh context, I think it's useful to have the error...)

The correct solution is to restrict access to use-lists of pure constants so that they don't affect program output. Then we wouldn't need to serialize or replicate them in any situation. At one point a number of years ago, I was only a few patches away from landing this...

dexonsmith · 2023-11-30T20:31:34Z

The correct solution is to restrict access to use-lists of pure constants so that they don't affect program output. Then we wouldn't need to serialize or replicate them in any situation. At one point a number of years ago, I was only a few patches away from landing this...

Found it:

https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606

This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. The current blocker is IROutliner has its own diy cloning, which doesn't use ValueMapper and it does make direct use of constant use lists. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes #58629

This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes #58629

Fix new test SimplifyLibCalls: Skip sincospi optimization for ConstantData Avoids looking at the uselist, and it would probably be more productive to constant fold this. PHITransAddr: Avoid looking at constant use lists Avoids asserts in GVN LICM: Avoid looking at use list of constant data The codegen test changes seem incidental. Either way, sms-grp-order.ll seems to already not hit the original issue. SCEVExpander: Don't look at uses of constants This could be more relaxed, and look for uses of globals in the same function but no tests apparently depend on that. IR: Remove uselist for constantdata This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes llvm#58629

This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes llvm#58629

This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes #58629

This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes #58629 Co-authored-by: Duncan P. N. Exon Smith <[email protected]>

This is a resurrected version of the patch attached to this RFC: https://discourse.llvm.org/t/rfc-constantdata-should-not-have-use-lists/42606 In this adaptation, there are a few differences. In the original patch, the Use's use list was replaced with an unsigned* to the reference count in the value. This version leaves them as null and leaves the ref counting only in Value. Remove use-lists from instances of ConstantData (which are shared across modules and have no operands). To continue supporting most of the use-list API, store a ref-count in place of the use-list; this is for API like Value::use_empty and Value::hasNUses. Operations that actually need the use-list -- like Value::use_begin -- will assert. This change has three benefits: 1. The compiler output cannot in any way depend on the use-list order of instances of ConstantData. 2. There's no use-list traffic when adding and removing simple constants from operand lists (although there is ref-count traffic; YMMV). 3. It's cheaper to serialize use-lists (since we're no longer serializing the use-list order of things like i32 0). The downside is that you can't look at all the users of ConstantData, but traversals of users of i32 0 are already ill-advised. Possible follow-ups: - Track if an instance of a ConstantVector/ConstantArray/etc. is known to have all ConstantData arguments, and drop the use-lists to ref-counts in those cases. Callers need to check Value::hasUseList before iterating through the use-list. - Remove even the ref-counts. I'm not sure they have any benefit besides minimizing the scope of this commit, and maintaining the counts is not free. Fixes llvm#58629 Co-authored-by: Duncan P. N. Exon Smith <[email protected]>

github-actions bot added the new issue label Oct 26, 2022

arsenm added llvm-tools All llvm tools that do not have corresponding tag tools:llvm-link llvm:bitcode llvm:asmparser labels Oct 26, 2022

EugeneZelenko removed the new issue label Nov 11, 2022

arsenm added the llvm-reduce label Apr 3, 2025

arsenm mentioned this issue Apr 7, 2025

IR: Remove uselist for constantdata #134692

Closed

arsenm mentioned this issue Apr 8, 2025

llvm-reduce should have a uselistorder reduction #134817

Open

arsenm mentioned this issue Apr 25, 2025

IR: Remove uselist for constantdata #137313

Merged

arsenm closed this as completed in #137313 May 6, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

uselistorder is broken for tools which load multiple files #58629

uselistorder is broken for tools which load multiple files #58629

arsenm commented Oct 26, 2022 •

edited by VoltrexKeyva

Loading

arsenm commented Nov 11, 2022

DaniilSuchkov commented Sep 18, 2023

dexonsmith commented Nov 30, 2023

dexonsmith commented Nov 30, 2023

uselistorder is broken for tools which load multiple files #58629

uselistorder is broken for tools which load multiple files #58629

Comments

arsenm commented Oct 26, 2022 • edited by VoltrexKeyva Loading

arsenm commented Nov 11, 2022

DaniilSuchkov commented Sep 18, 2023

dexonsmith commented Nov 30, 2023

dexonsmith commented Nov 30, 2023

arsenm commented Oct 26, 2022 •

edited by VoltrexKeyva

Loading