From af4469e5cfa65f2dfbeb221fe54933720a54707e Mon Sep 17 00:00:00 2001
From: Tim Hoffmann <2836374+timhoffm@users.noreply.github.com>
Date: Mon, 6 Apr 2020 00:21:27 +0200
Subject: [PATCH] Improve docs on automated tests

---
 doc/devel/coding_guide.rst | 39 +++++++++++++++++++++++++++++---------
 1 file changed, 30 insertions(+), 9 deletions(-)

diff --git a/doc/devel/coding_guide.rst b/doc/devel/coding_guide.rst
index ffac23812e95..422032dcef38 100644
--- a/doc/devel/coding_guide.rst
+++ b/doc/devel/coding_guide.rst
@@ -161,24 +161,45 @@ Merging
     A core dev should only champion one PR at a time and we should try to keep
     the flow of championed PRs reasonable.
 
+* Do not self merge, except for 'small' patches to un-break the CI or
+  when another reviewer explicitly allows it (ex, "Approve modulo CI
+  passing, may self merge when green").
+
 .. _pr-automated-tests:
 
 Automated tests
 ---------------
 
-* Make sure the Travis, Appveyor, CircleCI, and codecov tests are passing
-  before merging.
+Whenever a pull request is created or updated, various automated test tools
+will run on all supported platforms and versions of Python.
+
+* Make sure the Travis, Appveyor, CircleCI, and Azure pipelines are passing
+  before merging (All checks are listed at the bottom of the GitHub page of
+  your pull request). Here are some tips for finding the cause of the test
+  failure:
+
+  - If *Travis flake8* fails, you have a code style issue, which will be listed
+    near the bottom of the *Travis flake8* log.
+  - If any other Travis run fails, search the log for ``FAILURES``. The
+    subsequent section will contain information on the failed tests.
+  - If CircleCI fails, likely you have some reStructuredText style issue in
+    the docs. Search the CircleCI log for ``WARNING``.
+  - If Azure pipelines fail with an image comparison error, you can find the
+    images as *artifacts* of the Azure job:
 
-  - Whenever a pull request is created or updated, Travis and Appveyor
-    automatically runs the test suite on all versions of Python
-    supported by Matplotlib.  The tox_ support in Matplotlib may be
-    useful for testing locally.
+    - Click *Details* on the check on the GitHub PR page.
+    - Click *View more details on Azure Pipelines* to go to Azure.
+    - On the overview page *artifacts* are listed in the section *Related*.
+
+
+* Codecov and LGTM are currently for information only. Their failure is not
+  necessarily a blocker.
+
+* tox_ is not used in the automated testing. It is supported for testing
+  locally.
 
   .. _tox: https://tox.readthedocs.io/
 
-* Do not self merge, except for 'small' patches to un-break the CI or
-  when another reviewer explicitly allows it (ex, "Approve modulo CI
-  passing, may self merge when green").
 
 .. _pr-squashing: