You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
After #1671, the tests driven by testpythoneval.py will consist of type-checking sample programs and comparing their output to expected output.
This is the same job done by testcheck.py, but the latter runs most tests within the same process (with the exception of --incremental tests), which makes it much faster than testpythoneval.py which forks a mypy subprocess for each test case. As discussed in #1668, the pythoneval tests are by far the biggest contributor to the time it takes to run our tests, and they will probably still be the biggest after #1671.
So after #1671 is done, we can probably speed the tests up a lot by converting all the "pythoneval" test cases to the same format as the "check" test cases and using the latter driver for them. To really get the speedup, this might require a tweak to runtests.py to break up the "check" tests into several of its "tasks" -- just running it separately on each of its test-case files would probably do nicely.
This conversion should also simplify any future conversion to a standard test runner like py.test, by giving us one instead of two forms of these tests to convert.
The text was updated successfully, but these errors were encountered:
(Not really related, but which --incremental tests are running as a separate process? The code in testcheck.py runs them in the current process, like all other tests.)
Ah, thanks, I confused a detail between when I looked at the code yesterday
and when I wrote that comment. For --incremental we write the test
programs to separate files rather than pass them directly in the BuildSource, but we don't run a separate process. The separate file costs
something, but probably far less than the separate process.
After #1671, the tests driven by
testpythoneval.py
will consist of type-checking sample programs and comparing their output to expected output.This is the same job done by
testcheck.py
, but the latter runs most tests within the same process (with the exception of--incremental
tests), which makes it much faster thantestpythoneval.py
which forks a mypy subprocess for each test case. As discussed in #1668, the pythoneval tests are by far the biggest contributor to the time it takes to run our tests, and they will probably still be the biggest after #1671.So after #1671 is done, we can probably speed the tests up a lot by converting all the "pythoneval" test cases to the same format as the "check" test cases and using the latter driver for them. To really get the speedup, this might require a tweak to
runtests.py
to break up the "check" tests into several of its "tasks" -- just running it separately on each of its test-case files would probably do nicely.This conversion should also simplify any future conversion to a standard test runner like py.test, by giving us one instead of two forms of these tests to convert.
The text was updated successfully, but these errors were encountered: