Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add pylock parser and validator #13369

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 37 commits into
base: main
Choose a base branch
from
Draft

Conversation

sbidoul
Copy link
Member

@sbidoul sbidoul commented May 1, 2025

Towards #13334

TODO

  • add more tests
  • add more context to validation errors

@sbidoul sbidoul added the skip news Does not need a NEWS file entry (eg: trivial changes) label May 1, 2025
@sbidoul sbidoul force-pushed the pylock-parser_sbi branch 4 times, most recently from dbeaaab to 73f0c6a Compare May 3, 2025 10:43
@sbidoul sbidoul marked this pull request as ready for review May 3, 2025 10:45
@sbidoul
Copy link
Member Author

sbidoul commented May 3, 2025

Ok, this is in good shape, now.

Wondering if models.pylock is good enough to become a standalone lib...

@sbidoul sbidoul force-pushed the pylock-parser_sbi branch 2 times, most recently from 48627a9 to 7278c95 Compare May 3, 2025 12:26
@pfmoore
Copy link
Member

pfmoore commented May 3, 2025

Wondering if models.pylock is good enough to become a standalone lib...

... or maybe as part of the packaging library?

@sbidoul
Copy link
Member Author

sbidoul commented May 3, 2025

... or maybe as part of the packaging library?

Indeed, if we move out the toml export function (which I just did), it makes sense.

@sbidoul sbidoul force-pushed the pylock-parser_sbi branch 3 times, most recently from e5f7859 to 6ae54d3 Compare May 3, 2025 14:19
@sbidoul
Copy link
Member Author

sbidoul commented May 3, 2025

Ok, I went all the way to reading all the fields from the spec, and added a round trip test with the example from PEP 751.

@sbidoul sbidoul force-pushed the pylock-parser_sbi branch 4 times, most recently from 7e6d977 to 643f66f Compare May 3, 2025 21:21
@sbidoul
Copy link
Member Author

sbidoul commented May 3, 2025

Meh, kw_only dataclasses are cool, but do not exist in Python 3.9.

@sbidoul sbidoul force-pushed the pylock-parser_sbi branch 2 times, most recently from c3c068a to 51ddd3a Compare May 3, 2025 21:53
@sbidoul
Copy link
Member Author

sbidoul commented May 3, 2025

So I hand-knitted __init__ methods. It's a bit messy but that will be cleaned up when we drop Python 3.9 support.

Note, that all this is because I want to preserve the recommended field ordering from the spec for serialization, and there are required fields without default values that come after optional fields. Additionally keyword-only constructors make for a cleaner and more robust API.

@sbidoul sbidoul force-pushed the pylock-parser_sbi branch from fe115df to 33a6d0d Compare May 4, 2025 09:23
@sbidoul
Copy link
Member Author

sbidoul commented May 4, 2025

# type: ignore[type-abstract] relates to python/mypy#4717

@sbidoul sbidoul force-pushed the pylock-parser_sbi branch from 33a6d0d to f121187 Compare May 4, 2025 10:23
else:
output_file_path = Path(options.output_file)
if not is_valid_pylock_file_name(output_file_path):
if not is_valid_pylock_path(output_file_path):
logger.warning(
"%s is not a valid lock file name.",
output_file_path,
)
base_dir = output_file_path.parent
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I’m thinking, in many situations it might not actually be desirable to use a relative path. Not sure if there’s a good way to tell which way the user wants each package to be. Since outputting to stdout always generates an absolute path, maybe it’s less confusing if absolute path is the default when you output to a file too, and you need to somehow tell pip to use a relative path explicitly. Something similar to how we have --only-binary for install? I don’t know.

Copy link
Member Author

@sbidoul sbidoul May 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure either. I'd say relative paths below the pylock.toml location is a good default. And it gives a nice result for the common pip lock -e ..

Anyway, I'm going to remove this change from this PR and handle that separately.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel as though when you generate a pylock file, it makes sense for the paths to be relative to the location of the file. But when you generate to stdout (which has no location), it makes sense for paths to be absolute. As you say, though, maybe we don't want relative paths that go up in the filesystem hierarchy.

I'd go with:

  1. When writing to stdout, absolute.
  2. When writing to pylock, if the target is alongside (or below) the lockfile, then relative, ohterwise absolute.

But it's debatable enough that I think doing it as a separate PR is sensible.

sbidoul added 27 commits May 6, 2025 08:18
Move conversion of install requirements to pylock to utils namespace,
so model.pylock has no pip dependencies.
The context provided by the field name is sufficient, so
we can have the same error message for all package sources
We can remove them when we drop support for Python 3.9
Better start abstract and make it more concrete
later if/when use cases arises.
…ency-groups

The specs says these fields are optional with default = [].
But it also says "Tools SHOULD explicitly set this key to an empty array to signal that the inputs used to generate the lock file had no dependency groups (e.g. a pyproject.toml file had no [dependency-groups] table), signalling that the lock file is, in effect, multi-use even if it only looks to be single-use.", implying that
the distinction between absent and empty is meaningful.
@sbidoul sbidoul force-pushed the pylock-parser_sbi branch from e746ff4 to c3eceab Compare May 6, 2025 06:24
@sbidoul
Copy link
Member Author

sbidoul commented May 7, 2025

I proposed this to packaging in pypa/packaging#900

So marking as draft here until there is feedback there.

@sbidoul sbidoul marked this pull request as draft May 7, 2025 08:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
skip news Does not need a NEWS file entry (eg: trivial changes)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants