Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-153568: Share expression parsing with assignment targets - #157450

Open
pablogsal wants to merge 2 commits into
python:mainfrom
pablogsal:gh-153568-simplify-target-grammar
Open

gh-153568: Share expression parsing with assignment targets#157450
pablogsal wants to merge 2 commits into
python:mainfrom
pablogsal:gh-153568-simplify-target-grammar

Conversation

@pablogsal

@pablogsal pablogsal commented Sep 13, 2026

Copy link
Copy Markdown
Member

The grammar has two versions of almost the same parser: primary for expressions and t_primary for assignment and deletion targets. For factory().field[index], both walk through the call, attribute and subscript. When we try an assignment and then fall back to an expression, that means doing the same work through a different set of rules.

This lets targets use primary too. Parse the expression once, check that its shape is allowed as a target, then copy it into Store or Del context. For factory().field[index] = value, only the outer subscript changes context. The call, attribute and index stay Load, as does the cached expression.

That removes t_primary, star_atom, del_t_atom and the separate target list and tuple rules. It also folds the comparison forwarding rules into their parent, shares the keyword-argument prefix, and gives display items shorter names next to the list, tuple and set rules. There are 15 fewer grammar rules in total.

On my machine, this makes parsing the full standard library and twenty repositories about 9.1% faster overall.

Codebase Python files Parsing speed
Standard library 2,061 10.5% faster
Django 2,932 10.5% faster
pandas 1,527 9.4% faster
NumPy 495 9.8% faster
SciPy 999 9.8% faster
scikit-learn 1,036 10.0% faster
SymPy 1,607 7.8% faster
Transformers 4,884 8.7% faster
PyTorch 4,890 8.7% faster
Home Assistant 18,731 7.6% faster
mypy 442 9.5% faster
Requests 37 9.5% faster
Flask 83 9.0% faster
FastAPI 1,138 7.0% faster
SQLAlchemy 673 8.6% faster
Black 345 5.5% faster
pytest 274 9.2% faster
Ansible 1,853 8.9% faster
Scrapy 491 8.7% faster
Celery 428 10.4% faster
Sphinx 774 9.8% faster

The table compares this change against main at fb46c67d56, using release builds without PGO or LTO. Each result is the median of five paired runs. The timings include tokenization, building the AST and cleanup, without compiling bytecode or running the files.

Part of #153568.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant