Make testing improvements #3242
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 5 commits
April 26, 2024 16:20
fixes error when running tests using pytest -n >1 this fixes the with of the test time progress bars to avoid some failures in tests.
Tests from a loadgroup are all run sequentially on a single worker.
seems to miss marks in the conftest.py file itself though.
limit setup/teardown of visdom servers using a session scoped fixture. Add visdom tests to an xdist group to run them serially to avoid issues with server connection. add timeout to the tests explicitly requesting the server to further limit any future issues.
vfdev-5
reviewed
Apr 27, 2024
added 6 commits
April 29, 2024 09:53
Do not clean up visdom_server fixture. Session scoped fixtures are not guaranteed to be executed just once when using xdist and trying to cleanup twice can cause hangs. Timeout all visdom tests to avoid future issues with a hanging/dead server
vfdev-5
approved these changes
Apr 29, 2024
Collaborator
vfdev-5
left a comment
There was a problem hiding this comment.
Looks good, thanks John!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
Uses xdist loadgroups to improve performance of tests when running them on multiple workers. Setting
--dist=loadgroupswhen calling pytest should now send all tests with the "distributed" mark to the same worker. This helps to avoids some of the errors where tests are trying to use the resources made available by the distributed fixture. A similar solution is now implemented for tests using the visdom_server fixture.Also avoids the warnings for unregistered pytest marks and fixes some stray mypy failures.