From 232c1605a9b277c877e294746cf581fced73270b Mon Sep 17 00:00:00 2001 From: Stefanie Senger Date: Wed, 18 Sep 2024 11:23:35 +0200 Subject: [PATCH 1/4] [doc build] From f1ba873540d761cea84d8a387534df2a514e690b Mon Sep 17 00:00:00 2001 From: Stefanie Senger Date: Wed, 18 Sep 2024 13:24:07 +0200 Subject: [PATCH 2/4] add section on conflicting lock files --- doc/developers/contributing.rst | 52 +++++++++++++++++++++++++++++++-- doc/developers/tips.rst | 6 ---- 2 files changed, 49 insertions(+), 9 deletions(-) diff --git a/doc/developers/contributing.rst b/doc/developers/contributing.rst index 9ab3e99f9e51d..74e1394b2d1b1 100644 --- a/doc/developers/contributing.rst +++ b/doc/developers/contributing.rst @@ -556,12 +556,15 @@ Commit Message Marker Action Taken by CI Note that, by default, the documentation is built but only the examples that are directly modified by the pull request are executed. -Lock files -^^^^^^^^^^ +.. _build_lock_files: + +Build lock files +^^^^^^^^^^^^^^^^ CIs use lock files to build environments with specific versions of dependencies. When a PR needs to modify the dependencies or their versions, the lock files should be updated -accordingly. This can be done by commenting in the PR: +accordingly. This can be done by adding the following comment directly in the GitHub +Pull Request (PR) discussion: .. code-block:: text @@ -586,6 +589,49 @@ update documentation-related lock files and add the `[doc build]` marker to the @scikit-learn-bot update lock-files --select-build doc --commit-marker "[doc build]" +Resolve conflicts in lock files +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When there are conflicts between the lock files in a user's branch and the `main` +branch, the `main` branch must take precedence in defining the lock files. Only after +this is resolved, the changes from the user's PR can be reapplied on top of the updated +lock files. Therefore, conflicts should not be resolved directly on the GitHub PR +interface. Instead, follow these steps: + +Merge `upstream/main` into your local branch: + +.. prompt:: bash + + git pull upstream main --no-rebase + +Resolve conflicts, prioritizing the `upstream/main` branch. Be careful not to modify any +files marked with `# DO NOT EDIT`. These files are auto-generated and will be updated +in a later step. + +After resolving conflicts, add the changed files and proceed with the merge: + +.. prompt:: bash + + git add path/to/modified/files + git merge --continue + +Re-generate the environment and lock files. This can be done using one of the following +methods: + +- Use the scikit-learn-bot to update the lock files as described in the :ref:`build lock + files section `, or: + +- Run the python script to update the lock files manually: + + .. prompt:: bash + + python build_tools/update_environments_and_lock_files.py + + In order to run this script you might need to add `conda-lock` and `pip-tools` to your + environment as described on top of the file. + +Now you can add, merge and push the re-generated lock files back to your remote branch. + .. _stalled_pull_request: Stalled pull requests diff --git a/doc/developers/tips.rst b/doc/developers/tips.rst index 1c6ea5ba6f6f4..665c1aa0f71d5 100644 --- a/doc/developers/tips.rst +++ b/doc/developers/tips.rst @@ -218,12 +218,6 @@ PR-WIP: Regression test needed Please add a [non-regression test](https://en.wikipedia.org/wiki/Non-regression_testing) that would fail at main but pass in this PR. -PR-WIP: PEP8 - -:: - - You have some [PEP8](https://www.python.org/dev/peps/pep-0008/) violations, whose details you can see in the Circle CI `lint` job. It might be worth configuring your code editor to check for such errors on the fly, so you can catch them before committing. - PR-MRG: Patience :: From fce6a8e82818dc7af4f8126c96f6e5a94cfe7bfc Mon Sep 17 00:00:00 2001 From: Stefanie Senger Date: Mon, 18 Nov 2024 14:40:56 +0100 Subject: [PATCH 3/4] update after review --- doc/developers/contributing.rst | 47 +++++++++++++-------------------- 1 file changed, 18 insertions(+), 29 deletions(-) diff --git a/doc/developers/contributing.rst b/doc/developers/contributing.rst index c0921d77f21ad..80b97f1c10731 100644 --- a/doc/developers/contributing.rst +++ b/doc/developers/contributing.rst @@ -598,45 +598,34 @@ update documentation-related lock files and add the `[doc build]` marker to the Resolve conflicts in lock files ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -When there are conflicts between the lock files in a user's branch and the `main` -branch, the `main` branch must take precedence in defining the lock files. Only after -this is resolved, the changes from the user's PR can be reapplied on top of the updated -lock files. Therefore, conflicts should not be resolved directly on the GitHub PR -interface. Instead, follow these steps: - -Merge `upstream/main` into your local branch: +Conflicting lock files with the `upstream/main` branch can be resolved arbitrarily, and +we will re-generate the lock files afterwards. For instance, the conflicts can be +resolved as follows: .. prompt:: bash + # pull latest upstream/main git pull upstream main --no-rebase - -Resolve conflicts, prioritizing the `upstream/main` branch. Be careful not to modify any -files marked with `# DO NOT EDIT`. These files are auto-generated and will be updated -in a later step. - -After resolving conflicts, add the changed files and proceed with the merge: - -.. prompt:: bash - - git add path/to/modified/files + # resolve conflicts - keeping the upstream/main version for specific files + git checkout --theirs build_tools/*/*.lock build_tools/*/*environment.yml \ + build_tools/*/*lock.txt build_tools/*/*requirements.txt + git add build_tools/*/*.lock build_tools/*/*environment.yml \ + build_tools/*/*lock.txt build_tools/*/*requirements.txt git merge --continue -Re-generate the environment and lock files. This can be done using one of the following -methods: - -- Use the scikit-learn-bot to update the lock files as described in the :ref:`build lock - files section `, or: +This will merge `upstream/main` into our branch, automatically prioritising the +`upstream/main` for conflicting lock files. -- Run the python script to update the lock files manually: +If, after resolving the lock files, there are still conflicting files (for instance +dependency files), git will consider the merge incomplete and we resolve the remaining +conflicts as usual. - .. prompt:: bash +Finally, we have to re-generate the dependencies and lock files for the CIs, as described in +:ref:`Build lock files `, or by running: - python build_tools/update_environments_and_lock_files.py - - In order to run this script you might need to add `conda-lock` and `pip-tools` to your - environment as described on top of the file. +.. prompt:: bash -Now you can add, merge and push the re-generated lock files back to your remote branch. + python build_tools/update_environments_and_lock_files.py .. _stalled_pull_request: From d55605bfe46025ed27b943a5e7a6d46f97f7b960 Mon Sep 17 00:00:00 2001 From: Stefanie Senger Date: Fri, 22 Nov 2024 11:04:42 +0100 Subject: [PATCH 4/4] update --- doc/developers/contributing.rst | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/doc/developers/contributing.rst b/doc/developers/contributing.rst index 80b97f1c10731..3a939ee1be6e6 100644 --- a/doc/developers/contributing.rst +++ b/doc/developers/contributing.rst @@ -598,9 +598,7 @@ update documentation-related lock files and add the `[doc build]` marker to the Resolve conflicts in lock files ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Conflicting lock files with the `upstream/main` branch can be resolved arbitrarily, and -we will re-generate the lock files afterwards. For instance, the conflicts can be -resolved as follows: +Here is a bash snippet that helps resolving conflicts in environment and lock files: .. prompt:: bash @@ -608,20 +606,20 @@ resolved as follows: git pull upstream main --no-rebase # resolve conflicts - keeping the upstream/main version for specific files git checkout --theirs build_tools/*/*.lock build_tools/*/*environment.yml \ - build_tools/*/*lock.txt build_tools/*/*requirements.txt + build_tools/*/*lock.txt build_tools/*/*requirements.txt git add build_tools/*/*.lock build_tools/*/*environment.yml \ - build_tools/*/*lock.txt build_tools/*/*requirements.txt + build_tools/*/*lock.txt build_tools/*/*requirements.txt git merge --continue This will merge `upstream/main` into our branch, automatically prioritising the -`upstream/main` for conflicting lock files. +`upstream/main` for conflicting environment and lock files (this is good enough, because +we will re-generate the lock files afterwards). -If, after resolving the lock files, there are still conflicting files (for instance -dependency files), git will consider the merge incomplete and we resolve the remaining -conflicts as usual. +Note that this only fixes conflicts in environment and lock files and you might have +other conflicts to resolve. -Finally, we have to re-generate the dependencies and lock files for the CIs, as described in -:ref:`Build lock files `, or by running: +Finally, we have to re-generate the environment and lock files for the CIs, as described +in :ref:`Build lock files `, or by running: .. prompt:: bash