Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[ty] Compact retained semantic arrays#25454

Merged
charliermarsh merged 2 commits into
mainfrom
charlie/compact-retained-semantic-arrays
May 29, 2026
Merged

[ty] Compact retained semantic arrays#25454
charliermarsh merged 2 commits into
mainfrom
charlie/compact-retained-semantic-arrays

Conversation

@charliermarsh

@charliermarsh charliermarsh commented May 29, 2026

Copy link
Copy Markdown
Member

Summary

Reduces the memory retained by Salsa-cached semantic indexes by using a FrozenIndexVec for index-addressed immutable arrays, like the FrozenMap abstraction.

Apart from a >1% memory reduction, we see a 1% improvement on attrs, anyio, hydra-zen, multithreaded; 1% regression on pydantic and freqtrade.

@charliermarsh charliermarsh added the ty Multi-file analysis & type inference label May 29, 2026
@astral-sh-bot

astral-sh-bot Bot commented May 29, 2026

Copy link
Copy Markdown

Typing conformance results

No changes detected ✅

Current numbers
The percentage of diagnostics emitted that were expected errors held steady at 91.94%. The percentage of expected errors that received a diagnostic held steady at 87.09%. The number of fully passing files held steady at 92/134.

@astral-sh-bot

astral-sh-bot Bot commented May 29, 2026

Copy link
Copy Markdown

Memory usage report

Summary

Project Old New Diff Outcome
flake8 46.68MB 45.94MB -1.57% (752.19kB) ⬇️
trio 113.55MB 112.21MB -1.18% (1.34MB) ⬇️
sphinx 267.07MB 264.77MB -0.86% (2.31MB) ⬇️
prefect 721.11MB 714.89MB -0.86% (6.22MB) ⬇️

Significant changes

Click to expand detailed breakdown

flake8

Name Old New Diff Outcome
semantic_index 13.80MB 13.07MB -5.32% (752.19kB) ⬇️

trio

Name Old New Diff Outcome
semantic_index 29.86MB 28.52MB -4.49% (1.34MB) ⬇️

sphinx

Name Old New Diff Outcome
semantic_index 61.45MB 59.15MB -3.75% (2.31MB) ⬇️

prefect

Name Old New Diff Outcome
semantic_index 185.40MB 179.17MB -3.36% (6.22MB) ⬇️

@astral-sh-bot

astral-sh-bot Bot commented May 29, 2026

Copy link
Copy Markdown

ecosystem-analyzer results

No diagnostic changes detected ✅

Full report with detailed diff (timing results)

@astral-sh-bot

astral-sh-bot Bot commented May 29, 2026

Copy link
Copy Markdown

ruff-ecosystem results

Linter (stable)

✅ ecosystem check detected no linter changes.

Linter (preview)

✅ ecosystem check detected no linter changes.

@charliermarsh charliermarsh force-pushed the charlie/compact-retained-semantic-arrays branch from 0bfc0d5 to 55b7221 Compare May 29, 2026 14:06
@charliermarsh charliermarsh changed the base branch from charlie/intern to main May 29, 2026 14:06
@charliermarsh charliermarsh force-pushed the charlie/compact-retained-semantic-arrays branch from 55b7221 to bedff0d Compare May 29, 2026 14:17
@charliermarsh charliermarsh changed the title [ty] View memory benchmarks [ty] Compact retained semantic arrays May 29, 2026
@charliermarsh charliermarsh added the performance Potential performance improvement label May 29, 2026
@charliermarsh charliermarsh marked this pull request as ready for review May 29, 2026 15:56

@AlexWaygood AlexWaygood left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is very cool!

Comment thread crates/ty_python_core/src/use_def.rs Outdated
@charliermarsh charliermarsh force-pushed the charlie/compact-retained-semantic-arrays branch from bedff0d to 7982353 Compare May 29, 2026 16:33
@charliermarsh charliermarsh enabled auto-merge (squash) May 29, 2026 16:33
@charliermarsh charliermarsh force-pushed the charlie/compact-retained-semantic-arrays branch from 7982353 to 0d442ad Compare May 29, 2026 16:37

@MichaReiser MichaReiser left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is cool

Comment thread crates/ruff_index/src/frozen.rs Outdated
Comment thread crates/ruff_index/src/frozen.rs
Comment thread crates/ruff_index/src/frozen.rs
Comment thread crates/ruff_index/src/frozen.rs Outdated
Comment thread crates/ruff_index/src/frozen.rs Outdated
Comment thread crates/ruff_index/src/frozen.rs Outdated
impl<K: Ord, V> FromIterator<(K, V)> for FrozenMap<K, V> {
fn from_iter<T: IntoIterator<Item = (K, V)>>(iter: T) -> Self {
let mut entries = iter.into_iter().collect::<Vec<_>>();
entries.sort_unstable_by(|(left, _), (right, _)| left.cmp(right));

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you considered storing a keys and values Box? Would this make searching faster (because it needs to eat through less memory)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can try that!

@MichaReiser MichaReiser May 29, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to do this in a separate PR. It may also not be that impactful, given that the keys and values are all small types

Comment thread crates/ruff_index/src/frozen.rs Outdated
Comment thread crates/ruff_index/src/frozen.rs Outdated
@charliermarsh charliermarsh merged commit 572e4b5 into main May 29, 2026
60 checks passed
@charliermarsh charliermarsh deleted the charlie/compact-retained-semantic-arrays branch May 29, 2026 17:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

performance Potential performance improvement ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants