Thanks to visit codestin.com
Credit goes to github.com

Skip to content

refactor: add rule to do analysis time evaluation of environment markers #2832

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 28 commits into from
May 2, 2025

Conversation

rickeylev
Copy link
Collaborator

@rickeylev rickeylev commented Apr 27, 2025

wip/prototype to help bootstrap the impl of an analysis-time flag that evaluates
the pep508 dep specs

Creating a PR to make collab easier (maintainers can directly edit)

TODO:

  • Remove the todo markers after discussion

Work towards #2826

@rickeylev rickeylev requested review from aignas and groodt as code owners April 27, 2025 02:57
Copy link
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shall I push some of my thoughts as code or leave this at a comment level in the PR?

@rickeylev
Copy link
Collaborator Author

Shall I push some of my thoughts as code or leave this at a comment level in the PR?

Yes, go ahead and push changes.

Copy link
Collaborator Author

@rickeylev rickeylev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mapped in the various things. added 2 flags.

@rickeylev
Copy link
Collaborator Author

As I've poked around the various pep508.bzl files, I see there's a lot of little utilities, tests, edge cases, etc. I'm just plowing ahead and copy/pasting things. Feel free to point out specific places where something should be wired into something else

@rickeylev
Copy link
Collaborator Author

Ok, basic flag with tests is working.

Based on the convo in the other issue, it's pretty clear I didn't understand the jargon and renaming things is in order.

@rickeylev
Copy link
Collaborator Author

renamed to env_marker_setting, since it's all about evaluate env markers.

Saw this comment you added:

NOTE: Platform release for Android will be Android version:

I think there are android-specific flags in the android rules that could tell that info. However, we wouldn't want to depend on the android rules. If we want to cleanly support this, then a toolchain would probably be best. The way it could work is using toolchains in a plugin model type of way. Depending on the android rules runs their MODULE.bazel, which registers e.g. (toolchain_type=rules_python_platform_release, target_compatible_with=[android], target_settings=[@android//:version=X] value=X)

@rickeylev
Copy link
Collaborator Author

How is the extra env marker supposed to be evaluated? I'm having trouble finding an intelligible specification or example. All I've really found is extra == "pdf" as an example in pypa core metadata spec. So its a plain string? But there can be many extras? I don't get it.

@rickeylev
Copy link
Collaborator Author

Well, this started as a prototype, but it's actually functional now! PTAL

We could probably skip straight to analysis time dependency selection when the rest of the rest of the starlark pep 508 functionality is ready. I was trying to figure out how to re-activate those code paths for testing, but it wasn't immediately obvious.

@aignas
Copy link
Collaborator

aignas commented Apr 30, 2025

How is the extra env marker supposed to be evaluated? I'm having trouble finding an intelligible specification or example. All I've really found is extra == "pdf" as an example in pypa core metadata spec. So its a plain string? But there can be many extras? I don't get it.

Yeah, extra is just a string and it never is a version, so != and == in theory should be enough. However, expressions with extra should never passed to this marker_setting.

@aignas
Copy link
Collaborator

aignas commented Apr 30, 2025

Well, this started as a prototype, but it's actually functional now! PTAL

We could probably skip straight to analysis time dependency selection when the rest of the rest of the starlark pep 508 functionality is ready. I was trying to figure out how to re-activate those code paths for testing, but it wasn't immediately obvious.

Regarding this, I think you are right, we should go straight to analysis time. My thinking is that we need to have a list of deps that the hub repo has. Instead of target_platforms just have a list of whl names. Once we have Requires-Dist list, process extra conditions in the expressions and combine the expressions so that we have one expression per package. filter out the requires dist so that it only contains the dependencies from deps, so that bazel query still works. Generate extra targets using what is in this PR.

Comment on lines 43 to 44
"@platforms//os:freebsd": "freebsd",
"@platforms//os:openbsd": "openbsd",
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am reading it correctly, this value might be freebsd8 and I am wondering if we should do some trickery to combine this value with the platform_release?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs give "freebsd8" as an example from combining the uname calls. We don't know what freebsd version is being targeted, so I think it's best to just omit the version entirely.

re: combining with platform_release: platform_release is so poorly defined I don't think that's a good idea. In order to make it work, it'd have to be specific to a particular platform, i.e. the equivalent of "if //os:freebsd -> then concat('freebsd', $platform_release)". (Possible to express using select(), but if we go through that length, adding it to the toolchain is better IMHO).

The most correct thing would be to get the "undefined" cases from the toolchain; that's the best equivalent of "value when python was built".

@groodt
Copy link
Collaborator

groodt commented Apr 30, 2025

Yeah, extra is just a string and it never is a version, so != and == in theory should be enough. However, expressions with extra should never passed to this marker_setting

My 2c is that extras are more appropriate for to be passed as flags.

And markers are more appropriate to be mapped from platforms.

Extras are generally user provided and markers are generally build system / environment provided.

@rickeylev rickeylev changed the title wip: flag to evaluate pypa dep specs refactor: analysis time evaluation of environment markers Apr 30, 2025
@rickeylev rickeylev changed the title refactor: analysis time evaluation of environment markers refactor: add rule to do analysis time evaluation of environment markers May 1, 2025
@rickeylev
Copy link
Collaborator Author

I pulled out the newly added flags soas not to affect the public API for now. PTAL

I'll send a separate PR to factor out a common "env marker config" thing, as described in #2826 (comment)

Copy link
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM other than the map/value duplication, but that is reasonably easy to fix, I think.

Copy link
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since factoring out is in the separate PR. Feel free to do the value/map deduping in that PR as well.

@rickeylev rickeylev enabled auto-merge May 2, 2025 21:26
@rickeylev rickeylev added this pull request to the merge queue May 2, 2025
Merged via the queue into bazel-contrib:main with commit 8e76bd4 May 2, 2025
3 checks passed
@rickeylev rickeylev deleted the pep508.flag branch May 2, 2025 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants