Thanks to visit codestin.com
Credit goes to github.com

Skip to content

refactor(ast): migrate JavaScript string values to JSStr - #26548

Draft
Dunqing wants to merge 3 commits into
codex/jsstr-corefrom
codex/jsstr-migration
Draft

refactor(ast): migrate JavaScript string values to JSStr#26548
Dunqing wants to merge 3 commits into
codex/jsstr-corefrom
codex/jsstr-migration

Conversation

@Dunqing

@Dunqing Dunqing commented Sep 11, 2026

Copy link
Copy Markdown
Member

Decoded JavaScript strings currently use UTF-8 text plus separate lone-surrogate flags. Consumers can lose those flags or treat the internal escape encoding as literal text when constructing a new value.

Carry JSStr through decoded AST values, module requests, and their consumers, building on #26435. This is a Rust API change: callers that need UTF-8 must handle JSStr::as_str() returning None. Raw source text and identifiers keep their UTF-8 types. The draft includes enum evaluation and the associated owned UTF-16 storage; it does not introduce a general owned JSString.

Suggested review order

  1. Representation and parsing: AST string/cooked fields, lexer escape decoding, and parser recovery. Check that raw text stays distinct from decoded values and that surrogate pairs normalize across append boundaries.
  2. Output boundaries: oxc_str serialization, codegen, module-record conversion, and NAPI. Check JSON, raw-transfer, and lazy-transfer behavior, especially the Option<JSStr> niche and layout assertions.
  3. Enum evaluation: review the UTF-8/UTF-16 constant variants, surrogate-name hash table, scope separation, and cloning as one area.
  4. Other consumers: most linter edits adapt comparisons or explicitly reject values that UTF-8-only analysis cannot represent. String reconstruction, minifier folding, and JSX entity decoding need closer review than mechanical type updates.
  5. Generated output and regressions: read the generators before the generated diff. The js-str.test.ts files in the parser, minifier, and transformer bindings cover string values, module requests, recovery, and runtime behavior.

Deferred to a follow-up: simplify EscapeTable / StandardEscapeTable in oxc_estree into direct helpers. The migration retains their existing structure and eight-byte escape scanner while removing the obsolete marker decoder.

just ready passed after the latest commit, including workspace tests, Clippy, documentation, formatting, and generated-file checks; the working tree is clean. Additional validation covered parser conformance (no new failures), native parser/minifier/transformer tests, 3,723 runtime checks, and size/allocation tracking. Minifier size snapshots are unchanged. JavaScript lint and all 61 transformer binding tests also passed, and scalar lexer conformance regenerated only the expected fixture count. A direct Node/WASI smoke check covered module dependencies and enum string values. Browser execution has not been validated locally.

Part of #26242.

Implemented with AI assistance; draft for human review.

Decoded JavaScript strings need to preserve lone surrogates without
exposing their encoding as ordinary UTF-8 text. Carry JSStr through the
AST and its consumers so surrogate handling follows the value instead
of a separate flag.

Rust callers now explicitly handle values that cannot be borrowed as
str. Keep raw source text and identifiers in their UTF-8 types.

Part of #26242.

Implemented with AI assistance.

Dunqing commented Sep 11, 2026

Copy link
Copy Markdown
Member Author

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • 0-merge - adds this PR to the back of the merge queue
  • hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@github-actions github-actions Bot added A-linter Area - Linter A-parser Area - Parser A-semantic Area - Semantic A-cli Area - CLI A-minifier Area - Minifier A-ast Area - AST A-transformer Area - Transformer / Transpiler A-codegen Area - Code Generation A-isolated-declarations Area - Isolated Declarations A-ast-tools Area - AST tools A-formatter Area - Formatter A-linter-plugins Area - Linter JS plugins A-react-compiler Area - React Compiler A-ecmascript Area - ECMAScript labels Sep 11, 2026
@codspeed-hq

codspeed-hq Bot commented Sep 11, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 72 untouched benchmarks
⏩ 9 skipped benchmarks1


Comparing codex/jsstr-migration (5e2b1ec) with codex/jsstr-core (6610acd)

Open in CodSpeed

Footnotes

  1. 9 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Keep the existing escape-table structure while migrating string values
to JSStr so reviewers can focus on representation changes. The remaining
single-implementation abstraction can be removed in a follow-up.

Implemented with AI assistance.
Keep CI's generated-output checks in sync with the migration. The
transformer tests intentionally execute trusted fixture output to check
runtime string values, so allow that evaluation at those call sites.

Implemented with AI assistance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ast Area - AST A-ast-tools Area - AST tools A-cli Area - CLI A-codegen Area - Code Generation A-ecmascript Area - ECMAScript A-formatter Area - Formatter A-isolated-declarations Area - Isolated Declarations A-linter Area - Linter A-linter-plugins Area - Linter JS plugins A-minifier Area - Minifier A-parser Area - Parser A-react-compiler Area - React Compiler A-semantic Area - Semantic A-transformer Area - Transformer / Transpiler

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant