Can you update the readme based on the new functionality?

I'll try to take a look at the code ASAP.

Do you have any idea why there are so many more scikit-learn tests getting collected? What's special about the 12 tests in SciPy that don't get collected anymore?

All the code changes look reasonable to me and the new tests look like they're catching tricky corner cases along with just detecting that a module is imported.

Maybe add a test to make sure that tests still run in parallel if they don't use any functionality from a blacklisted module?

Can you update the readme based on the new functionality?

README updated.

Do you have any idea why there are so many more scikit-learn tests getting collected? What's special about the 12 tests in SciPy that don't get collected anymore?

The scikit-learn number was due to a bug after all. The inspect.isfunction returns False for methods so that thread safety inspection was stopping there.

What's special about the 12 tests in SciPy that don't get collected anymore?

The SciPy and scikit-learn that now get skipped were false positives. The ones I had a look at had calls to functions that were importing a different way than what we had blocklisted (e.g. from unittest.mock import patch) and/or tricky edge cases with methods that weren't analyzed before.

Thanks for looking at that! Making this stuff more robust will pay ecosystem-wide dividends.

Does adding the test I suggested in my other comment make sense?

Maybe add a test to make sure that tests still run in parallel if they don't use any functionality from a blacklisted module?

Isn't this covered from all of the other tests (in files other than tests/test_thread_unsafe_detection.py) that do not exercise this path?

Isn't this covered from all of the other tests (in files other than tests/test_thread_unsafe_detection.py) that do not exercise this path?

I don't think so. Or at least it's not obvious to me.

I think you have to add a new test to the test file created by test_thread_unsafe_ctypes that asserts num_parallel_threads is > 1 and doesn't actually use the ctypes module.

I think you have to add a new test to the test file created by test_thread_unsafe_ctypes that asserts num_parallel_threads is > 1 and doesn't actually use the ctypes module.

Done!

I spent some time profiling this and I didn't see any obvious way to speed things up - now that the visitor is doing more stuff, it takes more time. We also spend a decent amount of time in ast.parse and inspect.getsource - if there were faster alternatives that would also help, but I don't think there are any right now.

@rgommers what do you think about the marginally increased collection time for SciPy?

FWIW we inspect 3782 more functions on SciPy and 1748 more functions for scikit-learn. This probably means that the increased collection time depends on the structure of some functions as well, since scikit-learn tests get collected slightly faster even if more functions are analyzed. I can look deeper into this if we think the regression is significant enough.

That seems fine to me. 5 seconds more isn't a major concern. As long as we don't do that 10 more times and more than double the collection time:)

In that case, in it goes. Thanks for your work on this Lysandros!