TST: increase coverage on tk tests #22498
Conversation
1a03e1c to
153bfa1
Compare
| source = re.search(r"(?ms)^def .*", inspect.getsource(func)).group(0) | ||
|
|
||
| if "MPL_TEST_ESCAPE_HATCH" in os.environ: | ||
| return func |
There was a problem hiding this comment.
So, we are checking coverage of tests too? I guess I don't fully follow what coverage is reporting here...
There was a problem hiding this comment.
Yeah, we look at the coverage on the tests as well (@dopplershift has argued that tests are the only place insisting in 100% coverage actually make senses as if all of the code in your test suite is not running what are you even doing?!).
The change here is that instead of extracting the source and shoving it through python -c we are importing the test module and then running the actual test function in a sub-process. This bit of logic is so that when pytest discovers the tests it will go through the logic to spawn a sub-process to import an run the function but we import it to run it we need to actually run it. Setting an env in the subprocess call was the simplest way I found to do this.
There was a problem hiding this comment.
And I new see the warning that this is not actually covered and I am confused....
There was a problem hiding this comment.
(@dopplershift has argued that tests are the only place insisting in 100% coverage actually make senses as if all of the code in your test suite is not running what are you even doing?!).
There can be many reasons you don't run all of your test suite at a given time. Some optional dependencies may not be available, you could have very costly tests that you run only at larger intervals, ...
Out of curiosity, does coverage stretch across all the running CI jobs or where does it get its data from?
There was a problem hiding this comment.
yeah, at the end of all of our CI jobs we have something like
- name: Upload code coverage
uses: codecov/codecov-action@v2that pushes coverage information to codecov and then they merge all of them together.
If you click through the coverage UI enough you can get to https://codecov.io/gh/matplotlib/matplotlib/commit/153bfa18377c1a15a76906cfb77973971e0cee95/build which will show you which CI runs contributed to a given report.
| source = re.search(r"(?ms)^def .*", inspect.getsource(func)).group(0) | ||
|
|
||
| if "MPL_TEST_ESCAPE_HATCH" in os.environ: | ||
| return func |
There was a problem hiding this comment.
(@dopplershift has argued that tests are the only place insisting in 100% coverage actually make senses as if all of the code in your test suite is not running what are you even doing?!).
There can be many reasons you don't run all of your test suite at a given time. Some optional dependencies may not be available, you could have very costly tests that you run only at larger intervals, ...
Out of curiosity, does coverage stretch across all the running CI jobs or where does it get its data from?
|
🤦🏻 I cherry-picked this out wrong and lost the "give me more coverage" functionality. |
|
🤦🏻 I'm looking at the wrong local branch because work trees are fighting back...I need to finish my ☕ ..... |
Co-authored-by: Tim Hoffmann <[email protected]>
|
This should be ready to go again, the reason coverage was not showing up for the tests is that I had inverted the skip logic. I'll merged if/when CI runs and comes back green. |
PR Summary
This will increase the reported coverage of the tk tests, but shares commits
with #22005 and should be merged after that.
PR Checklist
Tests and Styling
pytestpasses).flake8-docstringsand runflake8 --docstring-convention=all).