Downloadable hash library in HTML summary #138
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
A HTML summary can be produced when generating
Include specific results from the image and hash test in the JSON summaries instead of infering them from RMS etc.
|
There are a lot of changes here, they look good though.
Can I suggest we put this next to the filter button (as an icon maybe?) I think it would be easier for sunpy contributors to find it if it wasn't behind another click? |
|
Sorry about all the changes. There was lots of changes which depended on other changes and I thought it would be quicker to merge them all together. I'll move the download to a "#" button to the left of the filter button on the top menu. |
|
Yeah, it's no worries, I just didn't do a line-by-line review ;) |
|
I moved the button: https://macbride.me/pytest-mpl/demo/html_generate_hashes/fig_comparison.html |
Cadair
approved these changes
Feb 21, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Main task was creating the hash library file in the results directory and making it available to download in the sort/filter menu in the HTML summary. Fixed various other things:
HTML summary when generating
When generating baseline images, the generated images now appear in a requested HTML summary. (See the "Generating baseline images..." demos: https://macbride.me/pytest-mpl/.)
Documented behaviour when generating hashes
Documented the existing behaviour: when generating a hash library, the tests will also be run as usual against the existing hash library specified by
--mpl-hash-libraryor the keyword argument. However, generating baseline images will always result in the tests being skipped.Make an updated hash library downloadable from HTML summary
When the
--mpl-results-alwaysoption is active, and some hash comparison tests are performed, a hash library containing all the result hashes will also be saved to the root of the results directory. The filename will be extracted from--mpl-generate-hash-library,--mpl-hash-libraryorhash_library=in that order.Made the generated hash library available to download in the sort/filter menu of the HTML summary. (See any demo involving hash generation or comparison: https://macbride.me/pytest-mpl/.)
Closes #135
Removed incorrect warning
Removed warning
Ignoring --mpl-result-path since --mpl-generate-path is setbecause when generating, a HTML summary may be saved there.Added tests for the generating options
These test that the JSON summaries match for various combinations of hashes and/or images generating options. Generated hash libraries are verified against the baseline. (Although, generated images are not currently verified.) Any generated HTML summaries are available here: https://macbride.me/pytest-mpl/
Additional commits on 2022-02-19
image_statusandhash_statusto the JSON report so it's easier to determine the individual results in the HTML report. Statuses arematch,diff,missing,generatedandNone.