Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-153568: Share expression parsing with assignment targets - #157450

Merged
pablogsal merged 2 commits into
python:mainfrom
pablogsal:gh-153568-simplify-target-grammar
Sep 14, 2026
Merged

gh-153568: Share expression parsing with assignment targets#157450
pablogsal merged 2 commits into
python:mainfrom
pablogsal:gh-153568-simplify-target-grammar

Conversation

@pablogsal

@pablogsal pablogsal commented Sep 13, 2026

Copy link
Copy Markdown
Member

The grammar has two versions of almost the same parser: primary for expressions and t_primary for assignment and deletion targets. For factory().field[index], both walk through the call, attribute and subscript. When we try an assignment and then fall back to an expression, that means doing the same work through a different set of rules.

This lets targets use primary too. Parse the expression once, check that its shape is allowed as a target, then copy it into Store or Del context. For factory().field[index] = value, only the outer subscript changes context. The call, attribute and index stay Load, as does the cached expression.

That removes t_primary, star_atom, del_t_atom and the separate target list and tuple rules. It also folds the comparison forwarding rules into their parent, shares the keyword-argument prefix, and gives display items shorter names next to the list, tuple and set rules. There are 15 fewer grammar rules in total.

On my machine, this makes parsing the full standard library and twenty repositories about 9.1% faster overall.

Codebase Python files Parsing speed
Standard library 2,061 10.5% faster
Django 2,932 10.5% faster
pandas 1,527 9.4% faster
NumPy 495 9.8% faster
SciPy 999 9.8% faster
scikit-learn 1,036 10.0% faster
SymPy 1,607 7.8% faster
Transformers 4,884 8.7% faster
PyTorch 4,890 8.7% faster
Home Assistant 18,731 7.6% faster
mypy 442 9.5% faster
Requests 37 9.5% faster
Flask 83 9.0% faster
FastAPI 1,138 7.0% faster
SQLAlchemy 673 8.6% faster
Black 345 5.5% faster
pytest 274 9.2% faster
Ansible 1,853 8.9% faster
Scrapy 491 8.7% faster
Celery 428 10.4% faster
Sphinx 774 9.8% faster

The table compares this change against main at fb46c67d56, using release builds without PGO or LTO. Each result is the median of five paired runs. The timings include tokenization, building the AST and cleanup, without compiling bytecode or running the files.

Part of #153568.

@bedevere-app bedevere-app Bot mentioned this pull request Sep 13, 2026
@pablogsal
pablogsal force-pushed the gh-153568-simplify-target-grammar branch from 8e86eef to d79c9b3 Compare September 14, 2026 15:45
@pablogsal
pablogsal enabled auto-merge (squash) September 14, 2026 15:45
@pablogsal
pablogsal force-pushed the gh-153568-simplify-target-grammar branch from d79c9b3 to 59e69f0 Compare September 14, 2026 16:10
@pablogsal
pablogsal merged commit c3f2373 into python:main Sep 14, 2026
54 checks passed
@pablogsal
pablogsal deleted the gh-153568-simplify-target-grammar branch September 14, 2026 16:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant