DOCS: include how to reproduce CI in PR template #17153

brunobeltran · 2020-04-16T04:13:05Z

PR Summary

Implement the suggestions to the PR checklist described in (https://discourse.matplotlib.org/t/how-to-engage-more-contributors-to-matplotlib/21059/11) and discussed in the last call.

Basically, gives users explicit instructions for how to "check" that each of the checkboxes is actually ready to check.

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

timhoffm

General structure

I find this list rather intimidating (this is not a comment on your PR but on the list in general). I'm always put off if I have to read through lengthy PR instructions.

I propose to create subsections for different aspects of PR like this:

General:

Code is Flake 8 compliant (run flake8 on changed files to check)
...

If new feature:

New features are documented, with examples if plot related
For major features add an entry to doc/users/next_whats_new/ (follow instructions in README.rst there)

If API changed:

Documented in doc/api/api_changes_[VERSION] if API changed in a
backward-incompatible way (follow instructions in README.rst there)

That way a user with simple bug fix or documentation fix doesn't have to read the new feature and API changed sections.

timhoffm · 2020-04-16T09:18:58Z

.github/PULL_REQUEST_TEMPLATE.md

+  [build](https://matplotlib.org/devel/documenting_mpl.html#building-the-docs)
+  without error)
+- [ ] New documentation conforms to matplotlib style conventions. (If you have
+  `flake8-docstrings` and `pydocstyle<4` installed, run


I think we can lift the restriction on pydocstyle<4. This was introduced in #14710 because of https://gitlab.com/pycqa/flake8-docstrings/issues/36 but is likely not neccessary anymore.

That bug was fixed, but the latest versions (I think pydocstyle>5.0.1) now crash on figure.py, as per PyCQA/pydocstyle#437. 4.0 also crashes and I didn't feel like chasing versions around so I left it at the one that I knew already worked...

brunobeltran · 2020-04-16T10:59:51Z

I agree that the list is pretty imposing.

I agree that splitting it up into sections would do a lot to help ameliorate number of brain cells required to figure out which of the check boxes apply to a PR. We could even include a comment that lets people know they can delete sections that don't apply to them.

I think the main thing that would make it less imposing would actually be changing the title. PR Checklist sounds like something that someone would be upset at me for messing up. Can't think of a good replacement right now but maybe something along the lines of "Useful PR Checks".

brunobeltran · 2020-04-16T11:19:02Z

For easy reading of the "rendered" version, here's my proposed PR template, after including @timhoffm's suggestions:

PR Summary

Summary would go here. Usually, only one or two of the check-box sections below would apply, and the rest could be omitted.

PR Checklists

For new code:

Has Pytest style unit tests (and pytest lib/matplotlib/tests passes)
Code is Flake 8 compliant (run flake8 on changed files to check)
New code is documented, with examples if plot related

For new documentation:

Documentation is sphinx and numpydoc compliant (the docs should build without error)
New documentation conforms to matplotlib style conventions. (If you have flake8-docstrings and pydocstyle<4 installed, run flake8 --docstring-convention=all on changed files to check).

For new features:

Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)

For API changes:

Documented in doc/api/api_changes_[VERSION] if API changed in a backward-incompatible way (follow instructions in README.rst there)

.github/PULL_REQUEST_TEMPLATE.md

story645

Thanks for putting this in & I wrote most of this language so I'm mostly critiquing my earlier work 🙃

.github/PULL_REQUEST_TEMPLATE.md

story645 · 2020-04-21T07:16:24Z

.github/PULL_REQUEST_TEMPLATE.md

+For new documentation:
+
+- [ ] Documentation is sphinx and numpydoc compliant (the docs should [build](https://matplotlib.org/devel/documenting_mpl.html#building-the-docs) without error)
+- [ ] New documentation conforms to matplotlib style conventions. (If you have `flake8-docstrings` and `pydocstyle<4` installed, run `flake8 --docstring-convention=all` on changed files to check).


Doesn't need the new (for consistency w/ above - if it's in the PR it should conform to the conventions)

I guess I worry that users will try to submit a simple PR and think they have to clean up warnings from an entire file, instead of just making sure that the docs they add/touch follow the conventions.

Although I guess it could be argued that if it fails flake8 --docstring-convention=all with our .flake8 file, it should be fixed, since we already explicitly ignore all "known" errors.

story645 · 2020-04-21T07:19:10Z

.github/PULL_REQUEST_TEMPLATE.md


 - Do not create the PR out of master, but out of a separate branch.

 - The PR title should summarize the changes, for example "Raise ValueError on
-  non-numeric input to set_xlim".  Avoid non-descriptive titles such as
+  non-numeric input to set_xlim". Avoid non-descriptive titles such as


Kind think maybe take out the descriptive pieces up top and replace w/ another checklist (folks seem to pay more attention to lists than to prose)
Pull Request
[] PR has descriptive name
[]PR has minimum of 1-2 sentence descriptive summary
[] optional:PR cross-links related issues

So I included a "Meta" section. To be honest, I think it's a little clearer just as prose, especially since it makes the already imposing list of checklists even longer....but if we have better luck with the checkboxes I guess you gotta do what you know works....

What about making the "Meta" section bullet points? This is more like a reminder. I don't want to tick numerous boxes when making a PR. These things are easy to apply and there is no reason not to immediately do it (in contrast to e.g. writing good tests, where sometimes it's reasonable to open a PR without.)

Same would maybe apply to "New Documentation" section?

I agree that I don't want to be checking off boxes for every PR when it's just things like "did you fill in the summary".

If I do make them just bullet points, then I see no need for it to ever be rendered (reviewer knows about these, don't need the visual noise), so I've changed it to be commented out by default.

Pasted latest version below.

brunobeltran · 2020-04-24T01:47:46Z

No description provided.

brunobeltran · 2020-04-24T01:50:05Z

PR Summary

PR Checklists

New code:

Has Pytest style unit tests (and pytest lib/matplotlib/tests passes)
Code is Flake 8 compliant (run flake8 on changed files to check)
Code is documented, with examples if plot related

New documentation:

Documentation is sphinx and numpydoc compliant (the docs should build without error)
Documentation conforms to matplotlib style conventions. (If you have flake8-docstrings and pydocstyle<4 installed, run flake8 --docstring-convention=all on changed files to check).

New features:

Have an entry in doc/users/next_whats_new/ (follow instructions in doc/users/next_whats_new/README.rst)

API changes:

Documented in doc/api/api_changes_[NEXT_VERSION] if API changed in a backward-incompatible way (follow instructions in doc/api/api_changes_[NEXT_VERSION]/README.rst)

.github/PULL_REQUEST_TEMPLATE.md

timhoffm · 2020-04-24T01:58:52Z

.github/PULL_REQUEST_TEMPLATE.md

+<!--
+Meta:
+
+- PR title summarizes the changes. (For example, prefer "Raise `ValueError` on non-numeric input to `set_xlim`" instead of "Addresses issue #8576").


Not sure how this long line will look. Is it better to hard-wrap after a fixed number of characters? (Also for other lines)

Yeah I think it makes sense to hard-wrap the comment lines. I played around with Github's current PR screen in dev mode and it has weird behavior on resizing but if we wrap at 79 characters we should be safe from having weird wrapping artifacts.

The checkbox lines interact funny with screen resizing because they're full of big "words" (e.g. doc/api/api_changes_[NEXT_VERSION]/README.rst), so I'd vote we leave them as single lines and let the Github text box wrap them for us "naturally".

If Github ever loses its mind and adds a horizontal slider to these boxes then we can come back and wrap those lines in the Markdown.

story645 · 2020-04-24T20:50:31Z

.github/PULL_REQUEST_TEMPLATE.md

+<!--
+Meta:
+
+- PR title descriptively summarizes the changes. (For example, prefer "Raise `ValueError` on non-numeric input to `set_xlim`" to "Addresses issue #8576").


Added some copyedits directly, but think since this is a bullet list now anyway:

PR title summarizes the changes. (Example: "Raise ValueError on non-numeric input set_xlim" )

Please don't use non-descriptive titles such as "Addresses issue support 'markevery' in prop_cycle #8576"

PR has at least a 1-2 sentence descriptive summary of how the PR implements the changes.

story645 · 2020-04-24T20:51:22Z

.github/PULL_REQUEST_TEMPLATE.md

+- [ ] Documented in `doc/api/api_changes_[NEXT_VERSION]` if API changed in a backward-incompatible way (follow instructions in `doc/api/api_changes_[NEXT_VERSION]/README.rst`)
+
+<!--
+Meta:


I'd pull this to the top of the template since it's the first thing they should do.

.github/PULL_REQUEST_TEMPLATE.md

brunobeltran · 2020-04-24T22:03:08Z

Thanks for the patience and great iterative feedback guys, I think we're getting close!

The organizational shift to have the section previously called "meta" at the top looks like it works really well, since it's now basically the first thing the user reads and avoids redundant language that way.

Decided to make all checkboxes sentence fragments (previously there was mis-mash of full sentences and fragments). Happy to instead go to all full sentences if this way looks silly to you guys, but figured they should all be the same, and this way is much shorter.

brunobeltran · 2020-04-24T22:04:32Z

PR Summary

PR Checklists

New code:

has Pytest style unit tests (and pytest lib/matplotlib/tests passes)
is Flake 8 compliant (run flake8 on changed files to check)
is documented, with examples if plot related

New documentation:

is sphinx and numpydoc compliant (the docs should build without error)
conforms to matplotlib style conventions (if you have flake8-docstrings and pydocstyle<4 installed, run flake8 --docstring-convention=all on changed files to check).

New features:

have an entry in doc/users/next_whats_new/ (follow instructions in doc/users/next_whats_new/README.rst)

API changes:

are documented in doc/api/api_changes_[NEXT_VERSION] if API changed in a backward-incompatible way (follow instructions in doc/api/api_changes_[NEXT_VERSION]/README.rst)

.github/PULL_REQUEST_TEMPLATE.md

QuLogic

Small typos and corrections.

.github/PULL_REQUEST_TEMPLATE.md

timhoffm

I'm torn with the first and last section. While they provide useful and comforting information, this makes the template rather long, which I personally find discouraging.

The alternative would be to drastically shorten and just link
https://matplotlib.org/devdocs/devel/coding_guide.html#summary-for-pr-authors.

.github/PULL_REQUEST_TEMPLATE.md

brunobeltran · 2020-04-27T16:06:28Z

@timhoffm, I actually agree that ideally this template would contain just the checklist (with no prose at the end). The actual prompt could be short and just read

<!-- 
Thank you for your PR! Please fill in this section with at least 1-2 sentences summarizing the purpose and implementation of your PR. 

Please take a moment to make sure your PR follows our [contributor guidelines]() before proceeding.
-->

But in order for that to work, IMO, we would need to be able to link a single page of the dev docs, and effectively have the checklist there. Currently, this is split over two places:

I think centralizing these instructions is enough work for a separate PR, and we have made two very concrete improvements in this new version:

The sections make it much easier to understand what parts of checklist that don't apply and delete unnecessary ones.
The instructions for how to check that CI will pass being here is almost more important than the actual description, IMO. ("Code conforms to style guide" is not as useful for a contributor as "flake8 --docstring-convention=all passes").

While I would love this PR to contain the shorter version of the template, pointing at the docs, I propose we merge this long version to get the benefits of the new additions, and when the docs have been centralized a bit (PR incoming), that PR can also shorten this template.

QuLogic · 2020-04-27T21:35:30Z

Can you drop this either on master of your fork, or in a small repo, so we can see how it looks when opening a PR?

brunobeltran · 2020-04-28T01:33:54Z

Pushed to my master branch, so you should be able to open a PR against brunobeltran:master, to see what this looks like.

QuLogic · 2020-05-05T00:58:32Z

For example, one might try going here and clicking Create pull request. For me, the template is 8.5-9 scroll lengths long. If I resize the entry as tall as possible, it is 3 and bit pages. This may be a bit too long, even though every subsection is short. I would need to compare it to other's to get a feel for that.

It's too bad GitHub doesn't have multiple PR templates like for issues.

Co-Authored-By: Elliott Sales de Andrade <[email protected]>

jklymak · 2020-08-24T15:08:45Z

A lot of work went into this. I'm somewhat against it, versus having the short checklist and a link to the actual docs explaining all this. But if those who like it actually like it, please merge. If you don't actually like it, lets close it?

brunobeltran · 2020-08-24T19:07:49Z

Thanks for bringing this to my attention again @jklymak.

I also think the bulk of the improvements that I'm starting to do here should go directly into the dev docs instead.

Closing in favor of #18341, which accomplishes the original goal without all the bloat that grew here (which I should also pull out into a proper PR of the dev docs).

timhoffm reviewed Apr 16, 2020

View reviewed changes

.github/PULL_REQUEST_TEMPLATE.md Outdated Show resolved Hide resolved

.github/PULL_REQUEST_TEMPLATE.md Outdated Show resolved Hide resolved

timhoffm added the Documentation label Apr 16, 2020

brunobeltran force-pushed the pr-template-improvements branch from ebbf32b to 2ed0b55 Compare April 17, 2020 00:21

story645 requested changes Apr 21, 2020

View reviewed changes

brunobeltran requested a review from story645 April 23, 2020 06:36

timhoffm reviewed Apr 24, 2020

View reviewed changes

story645 requested changes Apr 24, 2020

View reviewed changes

story645 approved these changes Apr 24, 2020

View reviewed changes

story645 reviewed Apr 24, 2020

View reviewed changes

.github/PULL_REQUEST_TEMPLATE.md Show resolved Hide resolved

QuLogic reviewed Apr 25, 2020

View reviewed changes

timhoffm reviewed Apr 25, 2020

View reviewed changes

.github/PULL_REQUEST_TEMPLATE.md Outdated Show resolved Hide resolved

brunobeltran force-pushed the pr-template-improvements branch 2 times, most recently from f49eb24 to 6587344 Compare April 27, 2020 16:09

brunobeltran force-pushed the pr-template-improvements branch 2 times, most recently from e845229 to d32f471 Compare April 28, 2020 01:30

brunobeltran added 4 commits July 22, 2020 12:19

DOCS: include how to reproduce CI in PR template

055b20c

DOCS: @timhoffm's PR template suggestions

57ab76b

DOCS: @story645's PR template suggestions

ff1a393

DOCS: don't show non-checkbox items in PR template

9498a34

brunobeltran and others added 4 commits July 22, 2020 12:20

DOCS: whats_new not just for "major feature"s

a94c061

mild copy edits

73fedf0

DOCS: pr template grammar and visual organization

55a0012

DOCS: @QuLogic's PR template suggestions

b2c6070

Co-Authored-By: Elliott Sales de Andrade <[email protected]>

brunobeltran force-pushed the pr-template-improvements branch from d32f471 to b2c6070 Compare July 22, 2020 19:20

brunobeltran mentioned this pull request Aug 24, 2020

DOCS: add action items to PR template #18341

Merged

brunobeltran closed this Aug 24, 2020

Uh oh!

DOCS: include how to reproduce CI in PR template #17153

DOCS: include how to reproduce CI in PR template #17153

Uh oh!

Conversation

brunobeltran commented Apr 16, 2020

PR Summary

PR Checklist

Uh oh!

timhoffm left a comment

Choose a reason for hiding this comment

General structure

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunobeltran Apr 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunobeltran commented Apr 16, 2020

Uh oh!

brunobeltran commented Apr 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklists

Uh oh!

Uh oh!

Uh oh!

story645 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunobeltran Apr 23, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

brunobeltran commented Apr 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brunobeltran commented Apr 24, 2020

PR Summary

PR Checklists

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

story645 Apr 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

brunobeltran commented Apr 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

brunobeltran commented Apr 24, 2020

PR Summary

PR Checklists

Uh oh!

Uh oh!

QuLogic left a comment

Choose a reason for hiding this comment

brunobeltran Apr 16, 2020 •

edited

Loading

brunobeltran commented Apr 16, 2020 •

edited

Loading

brunobeltran Apr 23, 2020 •

edited

Loading

brunobeltran commented Apr 24, 2020 •

edited

Loading

story645 Apr 24, 2020 •

edited

Loading

brunobeltran commented Apr 24, 2020 •

edited

Loading

brunobeltran commented Aug 24, 2020 •

edited

Loading