Thanks to visit codestin.com
Credit goes to github.com

Skip to content

DOC more updates to maintainers' doc #18954

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Dec 4, 2020
Merged

DOC more updates to maintainers' doc #18954

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
ec58d17
DOC more updates to maintainers' doc
ogrisel Dec 2, 2020
bc354dc
Update doc/developers/maintainer.rst
ogrisel Dec 2, 2020
58016c5
Update doc/developers/maintainer.rst
ogrisel Dec 2, 2020
efd3408
Apply suggestions from code review
ogrisel Dec 2, 2020
773fefa
Apply suggestions from code review
ogrisel Dec 2, 2020
f141f81
Improve structure of the maintainers doc
ogrisel Dec 2, 2020
4e973c7
Cosmetics
ogrisel Dec 2, 2020
d0e692b
update the checklist and cross-reference it
ogrisel Dec 2, 2020
ce10a7d
Refine checklist
ogrisel Dec 2, 2020
239d4b2
Fix bug in ordering
ogrisel Dec 2, 2020
a59b3bf
Move paragraph + more explicit branching commands
ogrisel Dec 2, 2020
73ac6f3
Apply suggestions from code review
ogrisel Dec 2, 2020
b3e86ec
Update doc/developers/maintainer.rst
ogrisel Dec 2, 2020
eaea674
Apply suggestions from code review
ogrisel Dec 3, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 140 additions & 57 deletions doc/developers/maintainer.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,81 +57,121 @@ permissions given to maintainers, which includes:
Preparing a release PR
......................

Releasing the first RC of e.g. version `0.99` involves creating the release
Major version release
~~~~~~~~~~~~~~~~~~~~~

Prior to branching please do not forget to prepare a Release Highlights page as
a runnable example and check that its HTML rendering looks correct. These
release highlights should be linked from the ``doc/whats_new/v0.99.rst`` file
for the new version of scikit-learn.

Releasing the first RC of e.g. version `0.99.0` involves creating the release
branch `0.99.X` directly on the main repo, where `X` really is the letter X,
**not a placeholder**. This is considered the *feature freeze*. The
development for the major and minor releases of 0.99 should
**also** happen under `0.99.X`. Each release (rc, major, or minor) is a tag
under that branch.
**not a placeholder**. The development for the major and minor releases of `0.99`
should **also** happen under `0.99.X`. Each release (rc, major, or minor) is a
tag under that branch.

This is done only once, as the major and minor releases happen on the same
branch:

.. prompt:: bash $

# Assuming upstream is an alias for the main scikit-learn repo:
git fetch upstream master
git checkout upstream/master
git checkout -b 0.99.X
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The explanation for this command now misses the key point that the release manager needs to do it either under a different clone from the main repo, or under the same clone, but from and to upstream instead of origin.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I expanded the commands to be more explicit.

git push --set-upstream upstream 0.99.X

Again, `X` is literal here, and `99` is replaced by the release number.
The branches are called ``0.19.X``, ``0.20.X``, etc.

In terms of including changes, the first RC ideally counts as a *feature
freeze*. Each coming release candidate and the final release afterwards will
include minor documentation changes and bug fixes. Any major enhancement or
feature should be excluded.
include only minor documentation changes and bug fixes. Any major enhancement
or feature should be excluded.

Then you can prepare a local branch for the release itself, for instance:
``release-0.99.0rc1``, push it to your github fork and open a PR **to the**
`scikit-learn/0.99.X` **branch**. Copy the :ref:`release_checklist` templates
in the description of the Pull Request to track progress.

This PR will be used to push commits related to the release as explained in
:ref:`making_a_release`.

You can also create a second PR from master and targeting master to increment
the ``__version__`` variable in `sklearn/__init__.py` to increment the dev
version. This means while we're in the release candidate period, the latest
stable is two versions behind the master branch, instead of one. In this PR
targeting master you should also include a new file for the matching version
under the ``doc/whats_new/`` folder so PRs that target the next version can
contribute their changelog entries to this file in parallel to the release
process.

Minor version release
~~~~~~~~~~~~~~~~~~~~~

The minor releases should include bug fixes and some relevant documentation
changes only. Any PR resulting in a behavior change which is not a bug fix
should be excluded.

First, create a branch, **on your own fork** (to release e.g. `0.999.3`):
First, create a branch, **on your own fork** (to release e.g. `0.99.3`):

.. prompt:: bash $

# assuming master and upstream/master are the same
git checkout -b release-0.999.3 master
git checkout -b release-0.99.3 master

Then, create a PR **to the** `scikit-learn/0.999.X` **branch** (not to
Then, create a PR **to the** `scikit-learn/0.99.X` **branch** (not to
master!) with all the desired changes:

.. prompt:: bash $

git rebase -i upstream/0.999.2
git rebase -i upstream/0.99.2

Do not forget to add a commit updating sklearn.__version__.
Copy the :ref:`release_checklist` templates in the description of the Pull
Request to track progress.

Do not forget to add a commit updating ``sklearn.__version__``.

It's nice to have a copy of the ``git rebase -i`` log in the PR to help others
understand what's included.

.. _making_a_release:

Making a release
................

0. Create the release branch on the main repo, if it does not exist. This is
done only once, as the major and minor releases happen on the same branch:

.. prompt:: bash $

git checkout -b 0.99.X

Again, `X` is literal here, and `99` is replaced by the release number.
The branches are called ``0.19.X``, ``0.20.X``, etc.
0. Ensure that you have checked out the branch of the release PR as explained
in :ref:`preparing_a_release_pr` above.

1. Update docs. Note that this is for the final release, not necessarily for
the RC releases. These changes should be made in master and cherry-picked
into the release branch, only before the final release.

- Edit the doc/whats_new.rst file to add release title and commit
statistics. You can retrieve commit statistics with:
- Edit the ``doc/whats_new/v0.99.rst`` file to add release title and list of
contributors.
You can retrieve the list of contributor names with:

::

$ git shortlog -s 0.99.33.. | cut -f2- | sort --ignore-case | tr '\n' ';' | sed 's/;/, /g;s/, $//'
$ git shortlog -s 0.98.33.. | cut -f2- | sort --ignore-case | tr '\n' ';' | sed 's/;/, /g;s/, $//' | fold -s

- For major releases, link the release highlights example from the ``doc/whats_new/v0.99.rst`` file.

- Update the release date in ``whats_new.rst``
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see a release date in the whats_new.rst file 😕.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We forgot on some of them in the past.


- Edit the doc/templates/index.html to change the 'News' entry of the front
page.
- Edit the ``doc/templates/index.html`` to change the 'News' entry of the
front page (with the release month as well).

2. On the branch for releasing, update the version number in
`sklearn/__init__.py`, the ``__version__`` variable by removing ``dev*``
only when ready to release. On master, increment the version in the same
place (when branching for release). This means while we're in the release
candidate period, the latest stable is two versions behind the master
branch, instead of one.

3. Proceed with caution. Ideally, tags should be created when you're almost
certain that the release is ready, since adding a tag to the main repo can
trigger certain automated processes. You can create a PR in the main repo
and trigger the wheel builder with the ``[cd build]`` commit marker using
``sklearn/__init__.py``, the ``__version__``.

For major releases, please add a 0 at the end: `0.99.0` instead of `0.99`.

For the first release candidate, use the `rc1` suffix on the expected final
release number: `0.99.0rc1`.

3. Trigger the wheel builder with the ``[cd build]`` commit marker using
the command:

.. prompt:: bash $
Expand All @@ -141,36 +181,70 @@ Making a release
The wheel building workflow is managed by GitHub Actions and the results be browsed at:
https://github.com/scikit-learn/scikit-learn/actions?query=workflow%3A%22Wheel+builder%22

Once all works, you can proceed with tagging. Create the tag and push it (if
it's an RC, it can be ``0.xxrc1`` for instance):
.. note::

Before building the wheels, make sure that the ``pyproject.toml`` file is
up to date and using the oldest version of ``numpy`` for each Python version
to avoid `ABI <https://en.wikipedia.org/wiki/Application_binary_interface>`_
incompatibility issues. Moreover, a new line have to be included in the
``pyproject.toml`` file for each new supported version of Python.
Comment on lines +184 to +190
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this information is not necessary if we decided to move on with oldest-supported-numpy #18900.


Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that we have to add a new step here (before step 4) to mention that the wheels should be uploaded to the staging area using the "Run workflow" form for the Wheel builder workflow.

Nevertheless, if the PR from step 3 is merged to the release branch, this step would not be necessary (i.f.f. the commit message contains [cd build] marker).

However, I do not really like the idea of merging an empty PR to the release brach.

.. note::

The acronym CD in `[cd build]` stands for `Continuous Delivery
<https://en.wikipedia.org/wiki/Continuous_delivery>`_ and refers to the
automation used to generate the release artifacts (binary and source
packages). This can be seen as an extension to CI which stands for
`Continuous Integration
<https://en.wikipedia.org/wiki/Continuous_integration>`_. The CD workflow on
GitHub Actions is also used to automatically create nightly builds and
publish packages for the developement branch of scikit-learn. See
:ref:`install_nightly_builds`.

4. Once all the CD jobs have completed successfully in the PR, merge it,
again with the `[cd build]` marker in the commit message. This time
the results will be uploaded to the staging area.

You should then be able to upload the generated artifacts (.tar.gz and .whl
files) to https://test.pypi.org using the "Run workflow" form for the
following GitHub Actions workflow:

https://github.com/scikit-learn/scikit-learn/actions?query=workflow%3A%22Publish+to+Pypi%22

5. If this went fine, you can proceed with tagging. Proceed with caution.
Ideally, tags should be created when you're almost certain that the release
is ready, since adding a tag to the main repo can trigger certain automated
processes.

Create the tag and push it (if it's an RC, it can be ``0.xx.0rc1`` for
instance):

.. prompt:: bash $

git tag -a 0.99 # in the 0.99.X branch
git push [email protected]:scikit-learn/scikit-learn.git 0.99
git tag -a 0.99.0 # in the 0.99.X branch
git push [email protected]:scikit-learn/scikit-learn.git 0.99.0

.. note::

Before building the wheels, make sure that the ``pyproject.toml`` file is
up to date and using the oldest version of ``numpy`` for each Python version
to avoid ABI incompatibility issues. Moreover, a new line have to be included
in the ``pyproject.toml`` file for each new supported version of Python.
6. Trigger the GitHub Actions workflow again but this time to upload the artifacts
to the real https://pypi.org (replace "testpypi" by "pypi" in the "Run
workflow" form).

4. Once the CD has completed successfully, collect the generated binary
wheel packages and upload them to PyPI by running the following commands
in the scikit-learn source folder (checked out at the release tag):
7. Alternatively, it's possible to collect locally the generated binary wheel
packages and source tarball and upload them all to PyPI by running the
following commands in the scikit-learn source folder (checked out at the
release tag):

.. prompt:: bash $

rm -r dist
pip install -U wheelhouse_uploader twine
python setup.py fetch_artifacts

This command will download all the binary packages accumulated in the `staging area on the anaconda.org
hosting service <https://anaconda.org/scikit-learn-wheels-staging/scikit-learn/files>`_ and put them in
your local `./dist` folder.
This command will download all the binary packages accumulated in the
`staging area on the anaconda.org hosting service
<https://anaconda.org/scikit-learn-wheels-staging/scikit-learn/files>`_ and
put them in your local `./dist` folder.

5. Check the content of the `./dist` folder: it should contain all the wheels
Check the content of the `./dist` folder: it should contain all the wheels
along with the source tarball ("scikit-learn-RRR.tar.gz").

Make sure that you do not have developer versions or older versions of
Expand All @@ -188,7 +262,7 @@ Making a release

twine upload dist/*

6. For major/minor (not bug-fix release), update the symlink for ``stable``
8. For major/minor (not bug-fix release), update the symlink for ``stable``
and the ``latestStable`` variable in
https://github.com/scikit-learn/scikit-learn.github.io:

Expand All @@ -206,11 +280,20 @@ Making a release
git commit -m "Update stable to point to 0.999"
git push origin master

.. _release_checklist:

Release checklist
.................

The following GitHub checklist might be helpful in a release PR::

* [ ] update news and what's new date in master and release branch
* [ ] create tag
* [ ] twine the wheels to PyPI when that's green
* [ ] update news and what's new date in release branch
* [ ] update news and what's new date and sklearn dev0 version in master branch
* [ ] check that the for the release wheels can be built successfully
* [ ] merge the PR with `[cd build]` commit message to upload wheels to the staging repo
* [ ] upload the wheels and source tarball to https://test.pypi.org
* [ ] create tag on the main github repo
* [ ] upload the wheels and source tarball to PyPI
* [ ] https://github.com/scikit-learn/scikit-learn/releases draft
* [ ] confirm bot detected at
https://github.com/conda-forge/scikit-learn-feedstock and wait for merge
Expand Down