Perf: AbortSignal-cancellable chunked scans for text-heavy rules (#150 Tier 3)#162
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
1 issue found.
About Unblocked
Unblocked has been set up to automatically review your team's pull requests to identify genuine bugs and issues.
📖 Documentation — Learn more in our docs.
💬 Ask questions — Mention @unblocked to request a review or summary, or ask follow-up questions.
👍 Give feedback — React to comments with 👍 or 👎 to help us improve.
⚙️ Customize — Adjust settings in your preferences.
…Tier 3) Adds lib/yielding-text-walk.ts: a chunked TreeWalker over text nodes that yields between chunks (default 100 nodes) via requestIdleCallback with setTimeout(0) fallback. Mirrors walkTextNodes' filter semantics (NON_CONTENT_TAGS, isInsidePlaceholder, minLength, shouldSkipParent). Sync fast path when the tree fits in one chunk — process and onComplete fire before the helper returns, so existing rule tests keep working without an explicit await. Each of the four text-heavy rules (pii-redact, secrets-redact, encoded-payload-redact, prompt-injection-redact) now owns a ReusableAbortController. Route-change events fire abortAndReset on each rule independently, so an in-flight chunked walk against the old tree stops cleanly when the SPA swaps to the new one — no more cross-tree mutations from a stale scan. Incremental subtree-watcher batches intentionally do NOT abort: they target their own scoped root and can complete concurrently with whatever else is running. Adds an abort-utils CJS stub via moduleNameMapper so the ESM-only package transparently loads in ts-jest's CJS pipeline. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
…boundaries
unblocked[bot] flagged a real bug. The consumer rules' `process`
callbacks call `replaceMatchesInTextNode`, which detaches each text
node in the chunk from the DOM — including the one walker.currentNode
points at. On resume, walker.nextNode() off a detached node returns
null and the walk silently ends after the first chunk.
Fix: pre-fetch the resume node before calling `process`, then anchor
walker.currentNode to it before yielding. Resume is held in a
`pendingResume` slot rather than seeded directly into the chunk —
seeding would skew the chunk-size invariant (property test caught
that variant on the first try).
Two regression tests:
- yielding-text-walk.test.ts: chunked walk where `process` detaches
each text node; the helper must still visit every node exactly
once.
- pii-redact.test.ts: 200-node tree, no abort, drain timers,
assert all 200 SSNs masked (not just chunk 1's 100).
Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
ace28e5 to
db14c11
Compare
CI's coverage threshold for ./src/rules/ requires 96% functions; the post-rebase build sat at 95.66% because the route-change callback each rule registers with subscribeRouteChange was never invoked by a test. Adds one "route change aborts the in-flight chunked walk" case per rule — mirror of the teardown abort test but the cancellation signal comes from a popstate event. Brings src/rules/ function coverage to 97%. Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements item 21 from issue #150.
Summary
The four text-heavy rules —
pii-redact,secrets-redact,encoded-payload-redact,prompt-injection-redact— used to scan largeinnerTextsynchronously, blocking the event loop for hundreds of ms on dense pages and producing a cross-tree mutation hazard when an SPA route swap landed mid-scan. This PR:lib/yielding-text-walk.ts— a chunked TreeWalker over text nodes that yields between chunks (default 100 nodes) and accepts an optionalAbortSignal. Yield mechanism isrequestIdleCallback(yields to layout/paint) with asetTimeout(0)fallback.ReusableAbortControllerfromabort-utils. Route-change events (via the existingsubscribeRouteChange) callabortAndReseton each rule independently, so an in-flight scan against the old tree stops cleanly when the SPA swaps to the new one.Incremental subtree-watcher batches do NOT abort — they target their own scoped root and don't conflict with whatever else is running. The structural-reset signal is the only cancellation channel, as discussed in #159's risk audit.
Design choices
chunkSize=100),processandonCompletefire before the helper returns. Existing rule tests keep working without explicitawaitmachinery — only the multi-chunk path (large pages, the abort tests) needs to flush microtasks.onCompletecallback instead of returningPromise<void>:prompt-injection-redactcollects containers in pass 1 then dedupes + hides in pass 2. The second pass must see the full container set, so it has to run after the last chunk —onCompletehandles that uniformly whether the walk was sync or chunked.process(chunk)detaches the chunk's text nodes (the consumer rules callreplaceMatchesInTextNode), so the walker'scurrentNodebecomes a detached node andnextNode()returns null. The fix pre-fetches the next node beforeprocessand re-anchorswalker.currentNodeto it before yielding. Caught by review feedback on the initial commit; regression tests added.src/__test-mocks__/abort-utils.ts, wired viamoduleNameMapper. The package is ESM-only and ts-jest's CJS pipeline can't transform it; the stub mirrors the runtime semantics each rule depends on (signal flips onabortAndReset,onAbortdispatch). Three pre-existing test files still carry inlinejest.mock("abort-utils", …)blocks that take precedence over the mapper — left in place to keep this PR focused.Rebase notes
Rebased onto
4a10459after #160 and #161 landed.Fix: prompt-injection-redact would re-hide a revealed container) touched the same hide loop this PR refactors. Reconciled theclosest(REVEALED_SELECTOR)skip check from Fix: prompt-injection-redact would re-hide a revealed container #161 by moving it into theonCompletecallback alongside the existingisInsidePlaceholder/isConnectedchecks — same semantic guard, just at the new location after the chunked-walk refactor. The 2 new tests Fix: prompt-injection-redact would re-hide a revealed container #161 added (reveal flowdescribe block) auto-merged into the test file and still pass.Fix: disguised-ad-flag re-hid the article a user just revealed) — no file overlap with this PR.Test plan
bun run test— 1285 / 1285 pass (4 new pass over pre-rebase baseline come from Fix: prompt-injection-redact would re-hide a revealed container #161's auto-merged tests)bun run typecheckbun run check(biome + eslint)bun run knipyielding-text-walk.test.ts): sync fast path, chunked path with microtask yield, signal handling (pre-aborted / mid-walk abort / mid-process abort), filter parity withwalkTextNodes(NON_CONTENT_TAGS, placeholder skip,minLength,shouldSkipParent), and the resume-anchor regression whereprocessdetaches each text node.yielding-text-walk.property.test.ts): for any tree + anychunkSize, the flat-mapped concatenation of all chunks equalswalkTextNodes's output in document order — covers sync large-chunk and async small-chunk paths plus the chunk-size invariant.prompt-injection-redactthe assertion is stronger —onCompleteis the hide pass, so an abort before it runs means no placeholders ever installed.pii-redactbuilds 200 SSNs, drains timers, asserts all 200 get masked (catches the resume-anchor regression end-to-end).🤖 Generated with Claude Code