Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[ty] Avoid retaining empty use-def tables#26018

Merged
charliermarsh merged 2 commits into
mainfrom
charlie/codex-sparse-use-def-data
Jun 17, 2026
Merged

[ty] Avoid retaining empty use-def tables#26018
charliermarsh merged 2 commits into
mainfrom
charlie/codex-sparse-use-def-data

Conversation

@charliermarsh

@charliermarsh charliermarsh commented Jun 15, 2026

Copy link
Copy Markdown
Member

Summary

We retain a UseDefMap for every scope. Several of its collection fields are empty in most scopes, but their fixed-size headers still remain in every map and add up across the semantic index.

This moves two sets of fields behind optional allocations. Predicate and constraint tables are omitted when no retained reachability or narrowing formula needs a table lookup. Bindings indexed by use, member states, and enclosing-scope snapshots share a second allocation that is omitted when all of those collections are empty.

Scopes without constraint tables use one shared empty value, preserving the existing lookup APIs without propagating optional values through evaluation. Predicates built for those scopes are also discarded because no retained constraint can reference them. The remaining fields share storage only to reduce the size of common UseDefMap values; they do not represent a new semantic abstraction.

The diagnostic comparison reports no changes.

Performance

The memory report shows total retained-memory reductions of 0.29–1.43% across Flake8, Trio, Sphinx, and Prefect. Within semantic_index, retained memory falls by 1.24–5.32%, saving between 491.55kB and 1.51MB.

@astral-sh-bot astral-sh-bot Bot added the ty Multi-file analysis & type inference label Jun 15, 2026
@astral-sh-bot

astral-sh-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 94.37%. The percentage of expected errors that received a diagnostic held steady at 89.00%. The number of fully passing files held steady at 94/134.

@astral-sh-bot

astral-sh-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
flake8 33.51MB 33.03MB -1.43% (491.55kB) ⬇️
trio 81.91MB 81.17MB -0.90% (752.78kB) ⬇️
sphinx 200.05MB 199.05MB -0.50% (1022.65kB) ⬇️
prefect 528.11MB 526.60MB -0.29% (1.51MB) ⬇️

Significant changes

Click to expand detailed breakdown

flake8

Name Old New Diff Outcome
semantic_index 9.03MB 8.55MB -5.32% (491.55kB) ⬇️

trio

Name Old New Diff Outcome
semantic_index 19.70MB 18.97MB -3.73% (752.78kB) ⬇️

sphinx

Name Old New Diff Outcome
semantic_index 40.51MB 39.51MB -2.47% (1022.65kB) ⬇️

prefect

Name Old New Diff Outcome
semantic_index 121.94MB 120.42MB -1.24% (1.51MB) ⬇️

@astral-sh-bot

astral-sh-bot Bot commented Jun 15, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

@charliermarsh charliermarsh force-pushed the charlie/codex-sparse-use-def-data branch 2 times, most recently from 97c4cc4 to 2c7a220 Compare June 15, 2026 21:33
@charliermarsh charliermarsh force-pushed the charlie/codex-sparse-use-def-data branch from 2c7a220 to 53c8775 Compare June 17, 2026 01:22
@charliermarsh charliermarsh changed the title [ty] Store sparse use-def data out of line [ty] Avoid retaining empty use-def tables Jun 17, 2026
@charliermarsh charliermarsh marked this pull request as ready for review June 17, 2026 01:53
@charliermarsh charliermarsh requested a review from a team as a code owner June 17, 2026 01:53
@charliermarsh charliermarsh added the performance Potential performance improvement label Jun 17, 2026
@astral-sh-bot astral-sh-bot Bot requested a review from sharkdp June 17, 2026 01:53
@charliermarsh charliermarsh requested review from MichaReiser and removed request for sharkdp June 17, 2026 01:54

@MichaReiser MichaReiser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recognize more and more Salsa patterns in ty's code ;)

narrowing_constraints: NarrowingConstraints,
/// Constraint lookup tables, absent when all retained constraints are built-in terminal
/// values that require no table lookup.
constraint_tables: Option<Box<ConstraintTables<'db>>>,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it common that UseDefMaps without a constraint_tables have an extra? Or what's the reasoning that this isn't part of extra?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so, because extra is always populated when you have a read given that it contains binding_by_use:

def f(x):
      y = x
      return y

(This is still empty in 65-75% of scopes though.)

@charliermarsh charliermarsh merged commit 77194b1 into main Jun 17, 2026
60 checks passed
@charliermarsh charliermarsh deleted the charlie/codex-sparse-use-def-data branch June 17, 2026 11:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Potential performance improvement ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants