[SimplifyCFG] Only consider provenance capture in store speculation #138548

nikic · 2025-05-05T16:06:44Z

The capture check here is to protect against concurrent accesses from other threads. This requires the provenance to escape.

llvmbot · 2025-05-05T16:07:24Z

@llvm/pr-subscribers-llvm-transforms

Author: Nikita Popov (nikic)

Changes

The capture check here is to protect against concurrent accesses from other threads. This requires the provenance to escape.

Full diff: https://github.com/llvm/llvm-project/pull/138548.diff

2 Files Affected:

(modified) llvm/lib/Transforms/Utils/SimplifyCFG.cpp (+3-1)
(modified) llvm/test/Transforms/SimplifyCFG/speculate-store.ll (+2-5)

diff --git a/llvm/lib/Transforms/Utils/SimplifyCFG.cpp b/llvm/lib/Transforms/Utils/SimplifyCFG.cpp
index 33af582a9e0a9..bde2e9d8ab8e3 100644
--- a/llvm/lib/Transforms/Utils/SimplifyCFG.cpp
+++ b/llvm/lib/Transforms/Utils/SimplifyCFG.cpp
@@ -3057,7 +3057,9 @@ static Value *isSafeToSpeculateStore(Instruction *I, BasicBlock *BrBB,
         Value *Obj = getUnderlyingObject(StorePtr);
         bool ExplicitlyDereferenceableOnly;
         if (isWritableObject(Obj, ExplicitlyDereferenceableOnly) &&
-            !PointerMayBeCaptured(Obj, /*ReturnCaptures=*/false) &&
+            capturesNothing(
+                PointerMayBeCaptured(Obj, /*ReturnCaptures=*/false,
+                                     CaptureComponents::Provenance)) &&
             (!ExplicitlyDereferenceableOnly ||
              isDereferenceablePointer(StorePtr, StoreTy,
                                       LI->getDataLayout()))) {
diff --git a/llvm/test/Transforms/SimplifyCFG/speculate-store.ll b/llvm/test/Transforms/SimplifyCFG/speculate-store.ll
index 22ffe213856b0..161ec380d31cf 100644
--- a/llvm/test/Transforms/SimplifyCFG/speculate-store.ll
+++ b/llvm/test/Transforms/SimplifyCFG/speculate-store.ll
@@ -203,11 +203,8 @@ define i32 @load_before_store_escape_addr_only(i64 %i, i32 %b)  {
 ; CHECK-NEXT:    [[ARRAYIDX:%.*]] = getelementptr inbounds [2 x i32], ptr [[A]], i64 0, i64 [[I:%.*]]
 ; CHECK-NEXT:    [[TMP0:%.*]] = load i32, ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[CMP:%.*]] = icmp slt i32 [[TMP0]], [[B:%.*]]
-; CHECK-NEXT:    br i1 [[CMP]], label [[IF_THEN:%.*]], label [[IF_END:%.*]]
-; CHECK:       if.then:
-; CHECK-NEXT:    store i32 [[B]], ptr [[ARRAYIDX]], align 4
-; CHECK-NEXT:    br label [[IF_END]]
-; CHECK:       if.end:
+; CHECK-NEXT:    [[SPEC_STORE_SELECT:%.*]] = select i1 [[CMP]], i32 [[B]], i32 [[TMP0]]
+; CHECK-NEXT:    store i32 [[SPEC_STORE_SELECT]], ptr [[ARRAYIDX]], align 4
 ; CHECK-NEXT:    [[TMP1:%.*]] = load i32, ptr [[A]], align 4
 ; CHECK-NEXT:    [[ARRAYIDX2:%.*]] = getelementptr inbounds [2 x i32], ptr [[A]], i64 0, i64 1
 ; CHECK-NEXT:    [[TMP2:%.*]] = load i32, ptr [[ARRAYIDX2]], align 4

dtcxzyw

LGTM.

dtcxzyw · 2025-05-06T13:20:04Z

llvm/lib/Transforms/Utils/SimplifyCFG.cpp

-            !PointerMayBeCaptured(Obj, /*ReturnCaptures=*/false) &&
+            capturesNothing(
+                PointerMayBeCaptured(Obj, /*ReturnCaptures=*/false,
+                                     CaptureComponents::Provenance)) &&


Similar pattern in LICM:

llvm-project/llvm/lib/Transforms/Scalar/LICM.cpp

Lines 2188 to 2200 in 6a99d81

// We know we can hoist the load, but don't have a guaranteed store.

// Check whether the location is writable and thread-local. If it is, then we

// can insert stores along paths which originally didn't have them without

// violating the memory model.

if (StoreSafety == StoreSafetyUnknown) {

Value *Object = getUnderlyingObject(SomePtr);

bool ExplicitlyDereferenceableOnly;

if (isWritableObject(Object, ExplicitlyDereferenceableOnly) &&

(!ExplicitlyDereferenceableOnly ||

isDereferenceablePointer(SomePtr, AccessTy, MDL)) &&

isThreadLocalObject(Object, CurLoop, DT, TTI))

StoreSafety = StoreSafe;

}

llvm-project/llvm/lib/Transforms/Scalar/LICM.cpp

Lines 1925 to 1954 in 6a99d81

bool isNotCapturedBeforeOrInLoop(const Value *V, const Loop *L,

DominatorTree *DT) {

// We can perform the captured-before check against any instruction in the

// loop header, as the loop header is reachable from any instruction inside

// the loop.

// TODO: ReturnCaptures=true shouldn't be necessary here.

return !PointerMayBeCapturedBefore(V, /* ReturnCaptures */ true,

L->getHeader()->getTerminator(), DT);

}

/// Return true if we can prove that a caller cannot inspect the object if an

/// unwind occurs inside the loop.

bool isNotVisibleOnUnwindInLoop(const Value *Object, const Loop *L,

DominatorTree *DT) {

bool RequiresNoCaptureBeforeUnwind;

if (!isNotVisibleOnUnwind(Object, RequiresNoCaptureBeforeUnwind))

return false;

return !RequiresNoCaptureBeforeUnwind ||

isNotCapturedBeforeOrInLoop(Object, L, DT);

}

bool isThreadLocalObject(const Value *Object, const Loop *L, DominatorTree *DT,

TargetTransformInfo *TTI) {

// The object must be function-local to start with, and then not captured

// before/in the loop.

return (isIdentifiedFunctionLocal(Object) &&

isNotCapturedBeforeOrInLoop(Object, L, DT)) ||

(TTI->isSingleThreaded() || SingleThread);

}

[SimplifyCFG] Only consider provenance capture in store speculation

49afe1b

The capture check here is to protect against concurrent accesses from other threads. This requires the provenance to escape.

nikic requested review from fhahn, aeubanks and dtcxzyw May 5, 2025 16:06

llvmbot added the llvm:transforms label May 5, 2025

dtcxzyw approved these changes May 6, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SimplifyCFG] Only consider provenance capture in store speculation #138548

[SimplifyCFG] Only consider provenance capture in store speculation #138548

nikic commented May 5, 2025

llvmbot commented May 5, 2025

dtcxzyw left a comment

dtcxzyw May 6, 2025

	// We know we can hoist the load, but don't have a guaranteed store.
	// Check whether the location is writable and thread-local. If it is, then we
	// can insert stores along paths which originally didn't have them without
	// violating the memory model.
	if (StoreSafety == StoreSafetyUnknown) {
	Value *Object = getUnderlyingObject(SomePtr);
	bool ExplicitlyDereferenceableOnly;
	if (isWritableObject(Object, ExplicitlyDereferenceableOnly) &&
	(!ExplicitlyDereferenceableOnly \|\|
	isDereferenceablePointer(SomePtr, AccessTy, MDL)) &&
	isThreadLocalObject(Object, CurLoop, DT, TTI))
	StoreSafety = StoreSafe;
	}

	bool isNotCapturedBeforeOrInLoop(const Value V, const Loop L,
	DominatorTree *DT) {
	// We can perform the captured-before check against any instruction in the
	// loop header, as the loop header is reachable from any instruction inside
	// the loop.
	// TODO: ReturnCaptures=true shouldn't be necessary here.
	return !PointerMayBeCapturedBefore(V, /* ReturnCaptures */ true,
	L->getHeader()->getTerminator(), DT);
	}

	/// Return true if we can prove that a caller cannot inspect the object if an
	/// unwind occurs inside the loop.
	bool isNotVisibleOnUnwindInLoop(const Value Object, const Loop L,
	DominatorTree *DT) {
	bool RequiresNoCaptureBeforeUnwind;
	if (!isNotVisibleOnUnwind(Object, RequiresNoCaptureBeforeUnwind))
	return false;

	return !RequiresNoCaptureBeforeUnwind \|\|
	isNotCapturedBeforeOrInLoop(Object, L, DT);
	}

	bool isThreadLocalObject(const Value Object, const Loop L, DominatorTree *DT,
	TargetTransformInfo *TTI) {
	// The object must be function-local to start with, and then not captured
	// before/in the loop.
	return (isIdentifiedFunctionLocal(Object) &&
	isNotCapturedBeforeOrInLoop(Object, L, DT)) \|\|
	(TTI->isSingleThreaded() \|\| SingleThread);
	}

[SimplifyCFG] Only consider provenance capture in store speculation #138548

Are you sure you want to change the base?

[SimplifyCFG] Only consider provenance capture in store speculation #138548

Conversation

nikic commented May 5, 2025

llvmbot commented May 5, 2025

dtcxzyw left a comment

Choose a reason for hiding this comment

dtcxzyw May 6, 2025

Choose a reason for hiding this comment