Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Ability to set the chunk size of a selectinload #11450

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
CaselIT opened this issue Jun 4, 2024 Discussed in #11448 · 2 comments
Open

Ability to set the chunk size of a selectinload #11450

CaselIT opened this issue Jun 4, 2024 Discussed in #11448 · 2 comments
Labels
loader options ORM options like joinedload(), load_only(), these are complicated and have a lot of issues orm PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers
Milestone
2.x.x

Comments

@CaselIT
Copy link
Member

CaselIT commented Jun 4, 2024

Discussed in #11448

Originally posted by ollz272 June 4, 2024
Hi,

We want to be able to customise the chunk size of selectin load. This is set on a code level at 500:

sqlalchemy/lib/sqlalchemy/orm/strategies.py

Line 2839 in 6f704ca

_chunksize = 500

We'd love to be able to change this, either as a project default, or on a case by case basis. Maybe something like:

selectinload(Table.relationship, chunk_size=1000)
selectinload(Table.relationship).options(chunk_size=1000)

If there are any other ways i'd love to hear.

See also mike's suggestions #11448 (comment) and #11448 (reply in thread)
@CaselIT CaselIT added orm loader options ORM options like joinedload(), load_only(), these are complicated and have a lot of issues PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers labels Jun 4, 2024
@CaselIT CaselIT added this to the 2.x.x milestone Jun 4, 2024
@ashm-dev
Copy link
Contributor

ashm-dev commented May 6, 2025

@CaselIT Can I take on this task?
What format should I use to transfer chunks?

@CaselIT
Copy link
Member Author

CaselIT commented May 6, 2025 •
edited
Loading

I think the first form selectinload(Table.relationship, chunk_size=1000) should be doable, since loader options already have an option dict that could be used to store this info, that's currently used for recursion_depth in selectinload, so it should be similar also for this new parameter.

We may want to validate this parameter to be > 0 if provided

@zzzeek do you have some additional suggestions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
loader options ORM options like joinedload(), load_only(), these are complicated and have a lot of issues orm PRs (with tests!) welcome a fix or feature which is appropriate to be implemented by volunteers
Projects
None yet
Milestone
2.x.x
Development

No branches or pull requests
2 participants
@CaselIT @ashm-dev