fix indeterministic markers #10667

radoering · 2025-12-20T14:41:00Z

Currently, this is only relevant for poetry-plugin-export because in poetry itself, we only evaluate the marker.

See python-poetry/poetry-plugin-export#334 (comment) ff.

Pull Request Check List

Added tests for changed code.
Updated documentation for changed code.

Summary by Sourcery

Ensure deterministic ordering of dependency groups and their markers when generating transitive package metadata.

Bug Fixes:

Fix non-deterministic marker generation by sorting dependency groups before combining group markers.

Enhancements:

Introduce and reuse a shared group sorting helper that prioritizes the main group for consistent output ordering.

Tests:

Update transitive package marker tests to use sets for groups and validate deterministic marker combinations regardless of input order.

sourcery-ai · 2025-12-20T14:41:07Z

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Makes environment marker derivation and serialization deterministic by consistently sorting groups (prioritizing 'main') and aligning tests with set-based group collections.

Sequence diagram for deterministic marker derivation and serialization

sequenceDiagram
    participant Locker
    participant TransitivePackageInfo
    participant group_sort_key
    participant Marker as BaseMarker

    Locker->>TransitivePackageInfo: get_marker(groups)
    activate TransitivePackageInfo
    TransitivePackageInfo->>group_sort_key: sort(groups, key=group_sort_key)
    group_sort_key-->>TransitivePackageInfo: (group != main, group) for each group
    TransitivePackageInfo->>Marker: union markers in sorted group order
    TransitivePackageInfo-->>Locker: combined BaseMarker
    deactivate TransitivePackageInfo

    Note over Locker,TransitivePackageInfo: In _dump_package
    Locker->>group_sort_key: sort(transitive_info.groups, key=group_sort_key)
    group_sort_key-->>Locker: (group != main, group) for each group
    Locker->>group_sort_key: sort(transitive_info.markers.items(), key=lambda x: group_sort_key(x[0]))
    group_sort_key-->>Locker: (group != main, group) for each marker group
    Locker->>Locker: serialize deterministic groups and markers

Class diagram for deterministic marker grouping and sorting

classDiagram
    class TransitivePackageInfo {
        int depth
        set~NormalizedName~ groups
        dict~NormalizedName, BaseMarker~ markers
        get_marker(groups Iterable_NormalizedName_) BaseMarker
    }

    class group_sort_key {
        <<function>>
        group_sort_key(group NormalizedName) tuple_bool_NormalizedName_
    }

    class BaseMarker {
    }

    class EmptyMarker {
    }

    TransitivePackageInfo ..> BaseMarker : uses
    TransitivePackageInfo ..> EmptyMarker : uses
    TransitivePackageInfo ..> group_sort_key : uses for sorting groups

    class Locker {
        _dump_package(package, transitive_info TransitivePackageInfo, data) void
    }

    Locker ..> TransitivePackageInfo : reads groups, markers
    Locker ..> group_sort_key : sorts groups and marker items

File-Level Changes

Change	Details	Files
Make transitive package marker combination order deterministic by sorting groups with a dedicated key function.	Introduce a reusable group_sort_key helper that orders groups with 'main' first, then lexicographically by name. Update TransitivePackageInfo.get_marker to iterate over groups using sorted(groups, key=group_sort_key) so marker unions follow a stable order.	`src/poetry/packages/transitive_package_info.py`
Ensure locker uses the same group sorting strategy when dumping package metadata and markers.	Replace inline lambda for sorting transitive_info.groups with the shared group_sort_key helper for consistent ordering between computation and serialization. Adjust sorting of transitive_info.markers.items() to use group_sort_key on the key part, keeping marker tables ordered deterministically.	`src/poetry/packages/locker.py`
Align tests with the new deterministic group handling by using sets to model groups.	Change parameterized test inputs for groups from lists to sets to better reflect unordered group collections and validate deterministic output under varying input order. Update test_get_marker type annotation from list[str] to set[str] to match the new set-based test inputs.	`tests/packages/test_transitive_package_info.py`

Possibly linked issues

#unknown: The PR fixes non-deterministic marker generation in lock/export, which causes the inaccurate markers reported in the issue.

Tips and commands

Interacting with Sourcery

Trigger a new review: Comment @sourcery-ai review on the pull request.
Continue discussions: Reply directly to Sourcery's review comments.
Generate a GitHub issue from a review comment: Ask Sourcery to create an
issue from a review comment by replying to it. You can also reply to a
review comment with @sourcery-ai issue to create an issue from it.
Generate a pull request title: Write @sourcery-ai anywhere in the pull
request title to generate a title at any time. You can also comment
@sourcery-ai title on the pull request to (re-)generate the title at any time.
Generate a pull request summary: Write @sourcery-ai summary anywhere in
the pull request body to generate a PR summary at any time exactly where you
want it. You can also comment @sourcery-ai summary on the pull request to
(re-)generate the summary at any time.
Generate reviewer's guide: Comment @sourcery-ai guide on the pull
request to (re-)generate the reviewer's guide at any time.
Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
pull request to resolve all Sourcery comments. Useful if you've already
addressed all the comments and don't want to see them anymore.
Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
request to dismiss all existing Sourcery reviews. Especially useful if you
want to start fresh with a new review - don't forget to comment
@sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

Enable or disable review features such as the Sourcery-generated pull request
summary, the reviewer's guide, and others.
Change the review language.
Add, remove or edit custom review instructions.
Adjust other review settings.

Getting Help

Contact our support team for questions or feedback.
Visit our documentation for detailed guides and information.
Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

sourcery-ai

Hey - I've reviewed your changes and they look great!

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

Currently, this is only relevant for poetry-plugin-export because in poetry itself, we only evaluate the marker.

This PR contains the following updates: | Package | Update | Change | |---|---|---| | [poetry](https://github.com/python-poetry/poetry) ([changelog](https://python-poetry.org/history/)) | minor | ` 2.2.1` -> `2.3.0` | --- ### Release Notes <details> <summary>python-poetry/poetry (poetry)</summary> ### [`v2.3.0`](https://github.com/python-poetry/poetry/blob/HEAD/CHANGELOG.md#230---2026-01-18) [Compare Source](python-poetry/poetry@2.2.1...2.3.0) ##### Added - **Add support for exporting `pylock.toml` files with `poetry-plugin-export`** ([#10677](python-poetry/poetry#10677)). - Add support for specifying build constraints for dependencies ([#10388](python-poetry/poetry#10388)). - Add support for publishing artifacts whose version is determined dynamically by the build-backend ([#10644](python-poetry/poetry#10644)). - Add support for editable project plugins ([#10661](python-poetry/poetry#10661)). - Check `requires-poetry` before any other validation ([#10593](python-poetry/poetry#10593)). - Validate the content of `project.readme` when running `poetry check` ([#10604](python-poetry/poetry#10604)). - Add the option to clear all caches by making the cache name in `poetry cache clear` optional ([#10627](python-poetry/poetry#10627)). - Automatically update the cache for packages where the locked files differ from cached files ([#10657](python-poetry/poetry#10657)). - Suggest to clear the cache if running a command with `--no-cache` solves an issue ([#10585](python-poetry/poetry#10585)). - Propose `poetry init` when trying `poetry new` for an existing directory ([#10563](python-poetry/poetry#10563)). - Add support for `poetry publish --skip-existing` for new Nexus OSS versions ([#10603](python-poetry/poetry#10603)). - Show Poetry's own Python's path in `poetry debug info` ([#10588](python-poetry/poetry#10588)). ##### Changed - **Drop support for Python 3.9** ([#10634](python-poetry/poetry#10634)). - **Change the default of `installer.re-resolve` from `true` to `false`** ([#10622](python-poetry/poetry#10622)). - **PEP 735 dependency groups are considered in the lock file hash** ([#10621](python-poetry/poetry#10621)). - Deprecate `poetry.utils._compat.metadata`, which is sometimes used in plugins, in favor of `importlib.metadata` ([#10634](python-poetry/poetry#10634)). - Improve managing free-threaded Python versions with `poetry python` ([#10606](python-poetry/poetry#10606)). - Prefer JSON API to HTML API in legacy repositories ([#10672](python-poetry/poetry#10672)). - When running `poetry init`, only add the readme field in the `pyproject.toml` if the readme file exists ([#10679](python-poetry/poetry#10679)). - Raise an error if no hash can be determined for any distribution link of a package ([#10673](python-poetry/poetry#10673)). - Require `dulwich>=0.25.0` ([#10674](python-poetry/poetry#10674)). ##### Fixed - Fix an issue where `poetry remove` did not work for PEP 735 dependency groups with `include-group` items ([#10587](python-poetry/poetry#10587)). - Fix an issue where `poetry remove` caused dangling `include-group` references in PEP 735 dependency groups ([#10590](python-poetry/poetry#10590)). - Fix an issue where `poetry add` did not work for PEP 735 dependency groups with `include-group` items ([#10636](python-poetry/poetry#10636)). - Fix an issue where PEP 735 dependency groups were not considered in the lock file hash ([#10621](python-poetry/poetry#10621)). - Fix an issue where wrong markers were locked for a dependency that was required by several groups with different markers ([#10613](python-poetry/poetry#10613)). - Fix an issue where non-deterministic markers were created in a method used by `poetry-plugin-export` ([#10667](python-poetry/poetry#10667)). - Fix an issue where wrong wheels were chosen for installation in free-threaded Python environments if Poetry itself was not installed with free-threaded Python ([#10614](python-poetry/poetry#10614)). - Fix an issue where `poetry publish` used the metadata of the project instead of the metadata of the build artifact ([#10624](python-poetry/poetry#10624)). - Fix an issue where `poetry env use` just used another Python version instead of failing when the requested version was not supported by the project ([#10685](python-poetry/poetry#10685)). - Fix an issue where `poetry env activate` returned the wrong command for `dash` ([#10696](python-poetry/poetry#10696)). - Fix an issue where `data-dir` and `python.installation-dir` could not be set ([#10595](python-poetry/poetry#10595)). - Fix an issue where Python and pip executables were not correctly detected on Windows ([#10645](python-poetry/poetry#10645)). - Fix an issue where invalid template variables in `virtualenvs.prompt` caused an incomprehensible error message ([#10648](python-poetry/poetry#10648)). ##### Docs - Add a warning about `~/.netrc` for Poetry credential configuration ([#10630](python-poetry/poetry#10630)). - Clarify that the local configuration takes precedence over the global configuration ([#10676](python-poetry/poetry#10676)). - Add an explanation in which cases `packages` are automatically detected ([#10680](python-poetry/poetry#10680)). ##### poetry-core ([`2.3.0`](https://github.com/python-poetry/poetry-core/releases/tag/2.3.0)) - Normalize versions ([#893](python-poetry/poetry-core#893)). - Fix an issue where unsatisfiable requirements did not raise an error ([#891](python-poetry/poetry-core#891)). - Fix an issue where the implicit main group did not exist if it was explicitly declared as not having any dependencies ([#892](python-poetry/poetry-core#892)). - Fix an issue where `python_full_version` markers with pre-release versions were parsed incorrectly ([#893](python-poetry/poetry-core#893)). </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).  Reviewed-on: https://git.walbeck.it/walbeck-it/docker-python-poetry/pulls/1654 Co-authored-by: renovate-bot <[email protected]> Co-committed-by: renovate-bot <[email protected]>

github-actions · 2026-01-22T00:38:45Z

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

sourcery-ai bot reviewed Dec 20, 2025

View reviewed changes

radoering mentioned this pull request Dec 20, 2025

marker ordering is nondeterministic python-poetry/poetry-plugin-export#334

Open

branchv previously approved these changes Dec 20, 2025

View reviewed changes

fix indeterministic markers

3a9b20d

Currently, this is only relevant for poetry-plugin-export because in poetry itself, we only evaluate the marker.

radoering dismissed branchv’s stale review via 3a9b20d December 21, 2025 06:02

radoering force-pushed the fix-indeterministic-marker branch from 4c4c5ea to 3a9b20d Compare December 21, 2025 06:02

branchv approved these changes Dec 21, 2025

View reviewed changes

radoering merged commit e8a05b7 into python-poetry:main Dec 22, 2025
54 checks passed

radoering mentioned this pull request Jan 17, 2026

release: bump version to 2.3.0 #10695

Merged

github-actions bot locked as resolved and limited conversation to collaborators Jan 22, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix indeterministic markers #10667

fix indeterministic markers #10667

radoering commented Dec 20, 2025 •

edited by sourcery-ai bot

Loading

Uh oh!

sourcery-ai bot commented Dec 20, 2025 •

edited

Loading

Reviewer's Guide

Sequence diagram for deterministic marker derivation and serialization

Class diagram for deterministic marker grouping and sorting

File-Level Changes

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Uh oh!

Uh oh!

github-actions bot commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

fix indeterministic markers #10667

fix indeterministic markers #10667

Conversation

radoering commented Dec 20, 2025 • edited by sourcery-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull Request Check List

Summary by Sourcery

Uh oh!

sourcery-ai bot commented Dec 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviewer's Guide

Sequence diagram for deterministic marker derivation and serialization

Class diagram for deterministic marker grouping and sorting

File-Level Changes

Possibly linked issues

Interacting with Sourcery

Customizing Your Experience

Getting Help

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions bot commented Jan 22, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

radoering commented Dec 20, 2025 •

edited by sourcery-ai bot

Loading

sourcery-ai bot commented Dec 20, 2025 •

edited

Loading