DOC: more work on release guide

tacaswell · tacaswell · commit 032b3555b5a4 · 2016-12-17T23:11:01.000-05:00
diff --git a/doc/devel/release_guide.rst b/doc/devel/release_guide.rst
@@ -1,174 +1,200 @@
+.. highlight:: bash
+
 .. _release-guide:
 
-*************
-Release Guide
-*************
+===============
+ Release Guide
+===============
 
 A guide for developers who are doing a matplotlib release.
 
-Development (beta/RC) release checklist
----------------------------------------
-
-- [ ] Testing
-- [ ] Create empty REL: commit
-- [ ] Create tag
-- [ ] Push tags + branches to GH
-- [ ] Notify mac/windows/conda builders
-- [ ] Merge everything to master
-- [ ] announce release
-
-Final release checklist
------------------------
-
-- [ ] Testing
-- [ ] update gh states
-- [ ] Create empty REL: commit
-- [ ] Create tag
-- [ ] Create branches
-- [ ] Push tags + branches to GH
-- [ ] release / DOI management
-- [ ] Notify mac/windows/conda builders
-- [ ] Merge everything to master
-- [ ] build documentation
-- [ ] upload to pypi / sf
-- [ ] deploy updated documentation
-- [ ] announce release
-
-Details
--------
+All Releases
+============
 
 .. _release-testing:
 
 Testing
-=======
+-------
 
 We use `travis-ci <https://travis-ci.org/matplotlib/matplotlib>`__ for
 continuous integration.  When preparing for a release, the final
 tagged commit should be tested locally before it is uploaded::
 
    python tests.py --processes=8 --process-timeout=300
 
-In addition ::
+In addition the following two tests should be run and manually inspected::
 
-   python unit/memleak_hawaii3.py
-
-should be run to check for memory leaks.  Optionally, make sure ::
-
-   cd examples/tests/
+   python unit/memleak_hawaii3.py``
+   pushd examples/tests/
    python backend_driver.py
-
-runs without errors and check the output of the PNG, PDF, PS and SVG
-backends.
+   popd
 
 
 .. _release_ghstats:
 
 Github Stats
-============
+------------
 
-To make sure everyone's hard work gets credited, regenerate the github
-stats.  In the project root run ::
+We automatically extract github issue, PRs, and authors from the github via the API::
 
-  python tools/github_stats.py --since-tag $TAG --project 'matplotlib/matplotlib' --links > doc/users/github_stats.rst
+  python tools/github_stats.py --since-tag $TAG --project 'matplotlib/matplotlib' --links --milestone v2.0.0 > doc/users/github_stats.rst
 
+Review and commit changes.  Some issue/PR titles may not be valid rst (the most common issue is
+``*`` which is interpreted as unclosed markup).
 
-where `$TAG` is the tag of the last major release.  This will generate
-stats for all work done since that release.
+.. _release_chkdocs:
 
-- [ ] review and commit changes
-- [ ] check for valid rst
-- [ ] re-add github-stats link
+Check Docs
+----------
+
+Before tagging, make sure that the docs build cleanly ::
+
+  pushd doc
+  python make.py html pdf -n 16
+  popd
+
+After the docs are built, check that all of the links, internal and external, are still
+valid.  We use ``linkchecker`` for this, which has not been ported to python3 yet.  You will
+need to create a python2 enviroment with ``requests==2.9.0`` and linkchecker ::
+
+  conda create -p /tmp/lnkchk python=2 requests==2.9.0
+  source activate /tmp/lnkchk
+  pip install linkchecker
+  pushd docs/build/html
+  linkchecker index.html --check-extern
+
+Address any issues which may arise.  The internal links are checked on travis, this should only
+flag failed external links.
 
 .. _release_tag:
 
-Create Tag
-==========
+Create release commit and tag
+-----------------------------
+
+To create the tag, first create an empty commit with a very terse set of the release notes
+in the commit message ::
+
+  git commit --allow-empty
+
+and then create a signed, annotated tag with the same text in the body
+message ::
+
+  git tag -a -s v2.0.0
+
+which will prompt you for your gpg key password and an annotation.
+For pre releases it is important to follow :pep:`440` so than the
+build artifacts will sort correctly in pypi.  Finally, push the tag to github ::
 
-On the tip of the current branch::
+  git push -t DANGER v2.0.0
+
+Congratulations, the scariest part is done!
+
+To prevent issues with any down-stream builders which download the
+tarball from github it is important to move all branches away from the commit
+with the tag ::
 
   git commit --allow-empty
-  git tag -a -s v1.5.0
+  git push DANGER master
 
-The commit and tag message should be very terse release notes should look something
-like
+If this is a final release, also create a 'doc' branch (this is not
+done for pre-releases)::
 
-  REL: vX.Y.Z
+   git branch v2.0.0-doc
+   git push DANGER v2.0.0-doc
 
-  Something brief description
+and if this is a major or minor release, also create a bug-fix branch (a
+micro release will be cut off of this branch)::
 
-Development releases should be post-fixed with the proper string following
-`PEP 440<https://www.python.org/dev/peps/pep-0440/>`__ conventions.
+   git branch v2.0.x
+   git push DANGER v2.0.x
 
-.. _release-branching:
 
-Branching
-=========
 
-Once all the tests are passing and you are ready to do a release, you
-need to create a release branch.  These only need to be created when
-the second part of the version number changes::
+.. _release_tag:
 
-   git checkout -b v1.1.x
-   git push git@github.com:matplotlib/matplotlib.git v1.1.x
+Release Management / DOI
+------------------------
 
+Via the github UI (chase down link), turn the newly pushed tag into a
+release.  If this is a pre-release remember to mark it as such.
 
-.. _release-packaging:
+For final releases also get a DOI from `zenodo
+<https://zenodo.org/>`__ and edit :file:`doc/_templates/citing.html`
+with DOI link and commit to the VER-doc branch and push to github ::
 
-Packaging
-=========
+  git push DANGER v2.0.0-doc:v2.0.0-doc
 
-* Make sure the :file:`MANIFEST.in` is up to date and remove
-  :file:`MANIFEST` so it will be rebuilt by MANIFEST.in
+.. _release_bld_bin:
 
-* run `git clean` in the mpl git directory before building the sdist
+Building binaries
+-----------------
 
-* unpack the sdist and make sure you can build from that directory
+We distribute mac, windows, and many linux wheels as well as a source
+tarball via pypi.  Before uploading anything, contact the various
+builders.  Mac and manylinux wheels are built on travis
+.  You need to edit the
+:file:`.travis.yml` file and push to master of `the build
+project <https://github.com/MacPython/matplotlib-wheels>`__.
 
-* Use :file:`setup.cfg` to set the default backends.  For windows and
-  OSX, the default backend should be TkAgg.  You should also turn on
-  or off any platform specific build options you need.  Importantly,
-  you also need to make sure that you delete the :file:`build` dir
-  after any changes to :file:`setup.cfg` before rebuilding since cruft
-  in the :file:`build` dir can get carried along.
+Update the ``master`` branch (for pre-releases the ``devel`` branch)
+of the `conda-forge feedstock
+<https://github.com/conda-forge/matplotlib-feedstock>`__ via pull request.
 
-* On windows, unix2dos the rc file.
+If this is a final release the following down-steam packagers should be contacted:
 
-* We have a Makefile for the OS X builds in the mpl source dir
-  :file:`release/osx`, so use this to prepare the OS X releases.
+- debian
+- fedora
+- arch
+- gentoo
+- macports
+- homebrew
+- Christoph Gohlke
+- continuum
+- enthought
 
-* We have a Makefile for the win32 mingw builds in the mpl source dir
-  :file:`release/win32` which you can use this to prepare the windows
-  releases.
+This can be done ahead of collecting all of the binaries and uploading to pypi.
 
+.. _release_upload_bin:
 
-Update PyPI
-===========
+make distribution and upload to pypi / SF
+-----------------------------------------
 
-This step tells PyPI about the release and uploads a source
-tarball. This should only be done with final (non-release-candidate)
-releases, since doing so will hide any available stable releases.
+Once you have collected all of the wheels, generate the tarball ::
 
-You may need to set up your `.pypirc` file as described in the
-`distutils register command documentation
-<http://docs.python.org/2/distutils/packageindex.html>`_.
+  git checkout v2.0.0
+  git clean -xfd
+  python setup.py sdist
 
-Then updating the record on PyPI is as simple as::
+and copy all of the wheels into :file:`dist` directory.  You should use
+``twine`` to upload all of the files to pypi ::
 
-    python setup.py register
+   twine -s upload dist/matplotlib*tar.gz
+   twine upload dist/*whl
 
-This will hide any previous releases automatically.
+Congratulations, you have now done the second scariest part!
 
-Then, to upload the source tarball::
+Additionally, for a final release, upload all of the files to sourceforge.
 
-    rm -rf dist
-    python setup.py sdist upload
+.. _release_docs:
 
+Build and Deploy Documentation
+------------------------------
+
+To build the documentation you must have the tagged version installed, but
+build the docs from the ``ver-doc`` branch.  An easy way to arrange this is::
+
+  pip install matplotlib
+  pip install -r doc-requirements.txt
+  git checkout v2.0.0-doc
+  git clean -xfd
+  cd doc
+  pyhton make.py html latex -n 16
+
+which will build both the html and pdf version of the documentation.
 
-Build and deploy Documentation
-==============================
 
 The built documentation exists in the `matplotlib.github.com
-<https://github.com/matplotlib/matplotlib.github.com/>`_ repository.
+<https://github.com/matplotlib/matplotlib.github.com/>`__ repository.
 Pushing changes to master automatically updates the website.
 
 The documentation is organized by version.  At the root of the tree is
@@ -177,39 +203,48 @@ there are directories containing the documentation for older versions.
 The documentation for current master are built on travis and push to
 the `devdocs <https://github.com/matplotlib/devdocs/>`__ repository.
 These are available `matplotlib.org/devdocs
-<http://matplotlib.org/devdocs>`__.  There is a symlink directory
-with the name of the most recently released version that points to the
-root.  With each new release, these directories may need to be
-reorganized accordingly.  Any time these version directories are added
-or removed, the `versions.html` file (which contains a list of the
-available documentation versions for the user) must also be updated.
+<http://matplotlib.org/devdocs>`__.
 
+Assuming you have this repository checked out in the same directory as
+matplotlib ::
 
-In the matplotlib source repository, build the documentation::
+  cd ../matplotlib.github.com
+  mkdir 2.0.0
+  rsync -a ../matplotlib/doc/build/html/* 2.0.0
+  cp ../matplotlib/doc/build/html/Matplotlib.pdf 2.0.0
 
-  cd doc
-  python make.py html
-  python make.py latex
+which will copy the built docs over.  If this is a final release, also
+replace the top-level docs ::
 
-Then copy the build products into your local checkout of the
-`matplotlib.github.com` repository (assuming here to be checked out in
-`~/matplotlib.github.com`::
+  rsync -a 2.0.0/* ./
 
-  cp -r build/html/* ~/matplotlib.github.com
-  cp build/latex/Matplotlib.pdf ~/matplotlib.github.com
+You will need to manually edit :file:`versions.html` to show the last
+3 tagged versions.  Now commit and push everything to github ::
 
-Then, from the `matplotlib.github.com` directory, commit and push the
-changes upstream::
+  git add *
+  git commit -a -m 'Updating docs for v2.0.0
+  git push DANGER master
 
-  git commit -m "Updating for v1.0.1"
-  git push upstream master
+Congratulations you have now done the third scariest part!
 
+It typically takes about 5-10 minutes for github to process the push
+and update the live web page (remember to clear your browser cache).
 
 
 Announcing
-==========
+----------
+
+The final step is to announce the release to the world.  A short
+version of the release notes along with acknowledgments should be sent to
+
+- matplotlib-user@python.org
+- matplotlib-devel@python.org
+- matplotlib-announce@python.org
+
+For final releases announcements should also be sent to the
+numpy/scipy/jupyter mailing lists and python-announce.
 
-Announce the release on matplotlib-announce, matplotlib-users, and
-matplotlib-devel.  Final (non-release-candidate) versions should also
-be announced on python-announce.  Include a summary of highlights from
-the CHANGELOG and/or post the whole CHANGELOG since the last release.
+In addition, annoucments should be made on social networks (twitter,
+g+, FB).  For major release, numFOCUS should be contacted for
+inclusion in their news letter and maybe to have something posted on
+their blog.
