Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-102120: Added an iter function that doesn't cache #102128

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 11 commits into from
May 23, 2023
Merged

gh-102120: Added an iter function that doesn't cache #102128

merged 11 commits into from
May 23, 2023

Conversation

PurityLake
Copy link
Contributor

@PurityLake PurityLake commented Feb 21, 2023 •
edited
Loading

Refers to #102120

This is a proposed solution to the linked issue which is to provide a way to iterate over a tar object so that it doesn't cache.

My suggestion is to reuse the code from Tarfile._next() to yield a tarinfo each time instead of caching it. This way it doesn't take up a tonne of memory.

I am looking for feedback on this so I will mark this as a draft.

Edit:

As from suggestions a parameter called stream can be passed to the object which will make sure nothing is cached. I will add a code example later today.

@bedevere-bot bedevere-bot mentioned this pull request Feb 21, 2023
Closed
@ethanfurman
Copy link
Member

I would rather add a new keyword to the TarFile signature -- perhaps stream -- and update the relevent locations in the code to not update members.

@PurityLake
Copy link
Contributor Author

That would probably be better

@arhadthedev arhadthedev added the stdlib Python modules in the Lib dir label Feb 25, 2023
@PurityLake
Copy link
Contributor Author

I'll be adding some tests later today

@PurityLake PurityLake marked this pull request as ready for review March 8, 2023 20:08
@PurityLake PurityLake requested a review from ethanfurman as a code owner March 8, 2023 20:08
@PurityLake
Copy link
Contributor Author

I believe this is working as intended now. I'm open to criticism in the implementation

ethanfurman
ethanfurman reviewed May 23, 2023
View reviewed changes
ethanfurman
ethanfurman requested changes May 23, 2023
View reviewed changes
Copy link
Member

@ethanfurman ethanfurman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall, looks good. I missed feature-freeze, though, so this will have to go in 3.13. Update the 3.12s to 3.13s and I'll get it merged.

@bedevere-bot
Copy link

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@PurityLake
Copy link
Contributor Author

@ethanfurman I went ahead and did that, thanks for reviewing it. It's too bad it didn't make it in before feature freeze but oh well.

@PurityLake
Copy link
Contributor Author

I have made the requested changes; please review again

@bedevere-bot
Copy link

Thanks for making the requested changes!

@ethanfurman: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from ethanfurman May 23, 2023 20:15
@ethanfurman ethanfurman merged commit 50fce89 into python:main May 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ethanfurman ethanfurman ethanfurman approved these changes

Assignees
No one assigned
Labels
stdlib Python modules in the Lib dir
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@PurityLake @ethanfurman @bedevere-bot @arhadthedev