Fix: extend unicode-invisibles-strip to cover bypass code points#204
Merged
Conversation
A single soft-hyphen (U+00AD) or similar zero-width / filler code point inserted inside a trigger word defeats every other regex-based rule in the catalog — prompt-injection-redact, secrets-redact, pii-redact, confirmshame-sanitize, disguised-ad-flag. The verb the agent reads is still semantically intact, but `\b<verb>\b` no longer matches. Add the highest-impact carriers that have no legitimate prose use: U+00AD SOFT HYPHEN U+034F COMBINING GRAPHEME JOINER U+115F, U+1160 HANGUL CHOSEONG / JUNGSEONG FILLER U+3164 HANGUL FILLER U+FFA0 HALFWIDTH HANGUL FILLER U+E0100–U+E01EF Variation Selectors Supplement (steganographic shape) Deliberately keep preserving U+FE00–U+FE0F (emoji-variant selectors — stripping changes every visible emoji), U+2800 (Braille blank), and U+180B–U+180D (Mongolian variation selectors). The Variation Selectors Supplement range and U+034F are combining marks, so they live in their own alternation arm — `replaceAll` with the alternation is semantically identical to a single character class and satisfies Biome's `noMisleadingCharacterClass`.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes one item from the red-team audit in #203 (Critical, XS).
A single soft hyphen (U+00AD) — or any of the other code points added here — inserted inside a trigger word defeats every regex-based rule in the catalog:
prompt-injection-redact,secrets-redact,pii-redact,confirmshame-sanitize,disguised-ad-flag. The verb the agent reads viatextContentis still semantically intact, but\b<verb>\bno longer matches because the invisible breaks the word boundary.Adds the highest-impact carriers with no legitimate prose use:
Deliberately not stripped (legitimate rendering depends on them; documented in the regex comment):
Notes for reviewers
unicode-invisibles-stripis position 15 inRULES_TUPLE;prompt-injection-redactis position 5. On the initial pass, the injection rule scans before the invisibles strip runs, so the soft-hyphen-in-verb bypass is still effective against the first sweep. Subsequent mutations / route changes pick up the cleaned text. A separate PR could move the invisibles strip earlier in the tuple — happy to do that if reviewers agree.noMisleadingCharacterClassrejects them inside a character class, so they live in their own alternation arms. Semantically identical for a globalreplaceAll.Test plan
bun run test -- unicode-invisibles-strip— added one case per new code point group (soft hyphen mid-word, CGJ, Hangul fillers, variation selectors supplement) plus regression cases for the deliberately preserved code points (U+FE0F, U+2800).bun run preflight— codegen + Biome + ESLint + typecheck + knip + Jest 1504/1504 green.🤖 Generated with Claude Code