I am also seeing failures at commit 7b7e7fe in numpy/typing/tests/test_typing.py. The travis test installs numpy, then does mkdir -p empty; cd empty; python ../runtests.py Perhaps some of the data files are missing from the shipped wheel, or a package version is causing some test variants to be skipped? I see 50 dots in the successful run, so that implies 50 tests are being run, which is the same as my local run, but I get "36 failed, 14 passed, 1 warning in 203.63s"

@BvB93 ping

Locally I'm not seeing any typing-related errors when running either python runtests.py -t numpy/typing or the full test suit with mkdir -p empty; cd empty; python ../runtests.py. Am i missing something here?

python3.8 runtests.py -t numpy/typing fails for me. My Python version is 3.8.5.

Thanks for checking. It might be something from the package list in the details above. Locally I see these differences (all the others are the same, especially mypy*, typ*):

For reference (macOS, python 3.8.3):

Could any of you show me an example of these failed test runs?
I'm curious as to whether they contain any useful information.

It seems to be some failure in mypy serializing? here is the output of the tests. The first assertion error is on line 1635 of that output.

Yes, there is definitely something wrong with mypy itself here.
The AssertionError seems to be raised somewhere during the mypy.api() call, just before the actual typing tests themselves are run.

The lack of actual error messages isn't helping either...

>   ???
E   AssertionError

After clearing out my mypy cache I've finally managed to reproduce the issue, though only when running python ./runtests.py --mypy numpy. Apparently it's an unsolved bug related to import cycles (python/mypy#8481) and indeed, removing core from the imports seems to resolve the issue.

This is definitely something to keep an eye out for in the future, especially once annotations are added for public sub-modules.