Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-126992: Change pickle code to base 10 for load_long and load_int #127042

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Dec 11, 2024

Conversation

Legoclones
Copy link
Contributor

@Legoclones Legoclones commented Nov 19, 2024

To be in line with the documentation and sync behavior across all 3 pickle implementations, load_int() and load_long() are changed to have the base explicitly set to 10 (like in pickletools.py).

Copy link
Contributor

@skirpichev skirpichev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

But see the issue thread. The discrepancy could be easily solved without backward-incompatible changes.

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a NEWS entry: see https://devguide.python.org/ HOWTO (use blurb-it or blurb).

@Legoclones
Copy link
Contributor Author

Just added a NEWS entry - I hope I did it right

Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vstinner
Copy link
Member

@serhiy-storchaka: Would you mind to double check this change?

@serhiy-storchaka
Copy link
Member

I am not sure that this is an issue. Non-decimal integers was accepted always. It was not documented, but user code may depend on it. We may even make it an official feature in future.

@skirpichev
Copy link
Contributor

I am not sure that this is an issue.

There is still an inconsistency wrt pickle and pickletools.

It was not documented, but user code may depend on it.

What do you think on changing pickletools instead, as it was suggested in the issue thread?

@serhiy-storchaka
Copy link
Member

If consider pickletools as a debugging tool, making it more lenient is safer.

@Legoclones
Copy link
Contributor Author

Legoclones commented Nov 22, 2024

Yeah the problem right now is that pickletools is LESS lenient because it ONLY accepts base 10, so if we want to keep the any-base feature in pickle, we should change pickletools to accept more bases than 10.

@Legoclones
Copy link
Contributor Author

@serhiy-storchaka Would you like me to change my pull request to update pickletools.py to accept any base?

@serhiy-storchaka
Copy link
Member

Take your time. In the end, the solution proposed in this PR may be the right solution.

Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After research and discussion I am inclined to accept this change. But please add tests. Test that prefixes 0x, 0o and 0b are rejected, but leading 0s are accepted and interpreted as decimals. Add also pickletools tests.

@vstinner
Copy link
Member

How do you want to write tests? Does it require to serialize to pickle manually (reimplement pickle in tests)? I don't think that tests are needed.

@serhiy-storchaka
Copy link
Member

There are already many similar tests. They try to unpickle the manually prepared data.

I added tests. They fail without changes in this PR.

@serhiy-storchaka
Copy link
Member

@Legoclones, it seems you are new with Git and made a common error (don't be ashamed, it happened with me too) -- you added changes to your main branch and created a PR from it. It makes impossible to you to create new PRs -- they will include changes from this PR. The right way -- create a separate branch for every PR (see the devguide for details).

Now your fork is broken. To fix it, delete the main branch in your fork on GitHub and your local copy and checkout the main branch from the upstream. Read the devguide about configuring an upstream remote repository. Do this only after this PR have been merged.

@serhiy-storchaka serhiy-storchaka enabled auto-merge (squash) December 11, 2024 12:30
@vstinner
Copy link
Member

@serhiy-storchaka:

I added tests. They fail without changes in this PR.

Oh nice, I like your tests :-) I expected something way more complicated.

@serhiy-storchaka serhiy-storchaka merged commit ce76b54 into python:main Dec 11, 2024
41 checks passed
@Legoclones
Copy link
Contributor Author

@serhiy-storchaka I've had a busy week at school, so I haven't had time to write the tests yet. Thanks for doing that and getting it done. Also yeah, I realized I had committed my changes to main and not a separate branch, I'll have to recreate it.

The tests look good, thanks for working with me on it!

srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants