3.14 pytest issues: suppress sqlite3 warnings, add context managers to mp calls - #3719
3.14 pytest issues: suppress sqlite3 warnings, add context managers to mp calls#3719filefolder wants to merge 17 commits into
Conversation
|
OK this should be working now. I think it may be an issue with PROJ-- have just suppressed the sqlite warnings specific to the cartopy tests in the |
| # which python 3.14 no longer tolerates | ||
| with warnings.catch_warnings(): | ||
| warnings.simplefilter("ignore", ResourceWarning) | ||
| gc.collect() |
There was a problem hiding this comment.
this looks so weird.. calling gc.collect() and then it is emitting warnings too. I've seen some weird errors in pytest, so I assume it's one of those, do you have a link to an error on https://tests.obspy.org that gets fixed by this?
There was a problem hiding this comment.
yes here: #3718
i am not sure gc.collect() actually needed there in hindsight.
seems like fundamentally it is out of our control so just filtering the warnings may be required, but i honestly couldn't figure it out
There was a problem hiding this comment.
does appear that the gc was needed afterall
There was a problem hiding this comment.
link to actual test run: https://tests.obspy.org/155382/
|
see #3718 (comment) |
|
suspect the new FDSN bug is 3.14 being stricter about a missing pool.close() or pool.terminate() in mass downloader. may have to go through all these. |
|
I've tried locally with 3.14 though and could not replicate any of these.. have you been able to get these fails locally? |
I have not.. but I think the issue could be here: Which leaves the mp stuff hanging specifically in pytest due to the overlapping "signal" imports. I already had to change something else regarding this for the 1.5 release using some creative importing. Fortunately by wrapping the 3-4 multiprocessing calls in a context manager (now done here) the problem doesn't seem to fail in pytest anymore and is probably a bit more robust anyway. Have re-shuffled a bit and mostly am now just suppressing the specific sqlite3 pytest warnings in pytest.ini. There was also a new issue with py3.10 ubuntu mindep, the 5 minute timeout didn't seem to be enough. Have bumped to 10 and made the libmamba solver explicitly used. On that tangent maybe we should turn off the 3.8 and 3.9 tests now? |
|
Looking at this again, I think it should go into maintenance branch? I've merged maintenance into master, so the sqlalchemy fixes are in there now and I will retrigger tests here now to see what the status is. |
|
so the thing with "signal" being shadowed by obspy.signal is solved in #3789 - leaving this open for the other issues, we need to check if they still come up. |
python3.14 causing various test issues, some can be fixed on our end, others probably have to be ignored
multiprocessing causing issues in pytest, solution is mostly to wrap in context managers and/or use threadpool.
there are some places where SQLite databases aren't closing which 3.14 no longer tolerates. seems to only be a problem in pytest, solution is probably just to ignore these warnings and/or update pytest-cov.
increased timeout buffer for py3.10 mindep ubuntu to 10 minutes (5 wasn't cutting it), force use of the faster libmamba solver which it didn't appear to be using.
based on master