-
Notifications
You must be signed in to change notification settings - Fork 5.7k
Refactor and overhaul the test suite #3426
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This commit additionally changes: - The scope of the fixtures to module - Class variables are moved into a namespace class called as Space - Auto addition of flaky and no_req marker (in conftest.py) - Caching of default and tz bots (in conftest.py) Apart from this, no tests were added/deleted/modified in any way.
Also adapt the bot tests for this change + removes debug prints + fix few tests
Also change the scope of some fixtures
This mainly does two things: - Leverages the session bot while making new applications, this eliminates all calls to get_me - `test_signal_handler` was optimized to use `loop.call_later` which eliminates the chance where we could get a TaskException not retrieved error, which often fatally crash pytest. These two changes thus reduce flakyness and improves speed.
The last commit addresses the root cause of its addition, so now we don't need this :)
These tests don't fail for me locally, will have a look later. I suspect it is due to the change of scope of those fixtures. Edit: See commit message below for details on why it failed. |
This was done as when running pytest with -x, this error is emitted. Not doing this seems to fix the problem. However this could backfire on python versions older than 3.10.6
this is based on whether the test calls get_me indirectly or not
β¦tinefunction The limitation of that function is fixed by using the `__wrapped__` attribute
β¦d/create_task let's take full advantage of the lib now being async
includes removing unused params, removing unnecessary monkeypatch.delattr's, missing import, beautification, etc
β¦llback tests This means we can now move those methods back to TestBotNoReq, since they did not make any request apart from delete_webhook()
The tests are roughly arranged as follows: - test_slot_behaviour - test de_json/to_dict - test init - test equality - test wrong init - etc
Also add a missing test_to_dict in user.py
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I reviewd the commits since my last review. In a few places it looks like you duplicated an already exsting tests. Maybe those are merge errors or maybe it only looks that way if you don't view all commits β¦
I think it looks like that since I had to rearrange those tests into these classes. Nevertheless i'll check again |
specially on python 3.7. The xdist_group markers are kept so we can use them locally
Also increase loop.call_later call by 0.1 seconds in test_signal_handlers
Awesome work! πͺ |
Closes #3407. I did not include the
req
marker, but if you'd like that can be easily added as well.Also some new "features":
Dict(Ext)Bot
now overridesget_me
to return a cachedUser
object, which is obtained from thebot_info
fixture. This saves lots of requests to the realget_me
every time we initialize a bot, app, or updater.Many tests were optimized as well, which should decrease the flakyness in the CI.
How do I review this?!
It is best to review by commit, the changes are specific and I rebased multiple times to make sure changes fit in a certain category.