fix: pytest plugin now skips all tests in unaffected files #885
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously, only the first test from each unaffected file was skipped. This was due to the `if item.path in seen: continue` check skipping the removal logic for subsequent tests in the same file. The fix uses a two-pass approach: 1. First pass identifies all paths that should be removed 2. Second pass filters out all items from those paths at once Before: 8 tests collected, 1 skipped, 7 passed (bug) After: 8 tests collected, 8 skipped, no tests ran (correct)
DetachHead
reviewed
Dec 24, 2025
Collaborator
DetachHead
left a comment
There was a problem hiding this comment.
thanks for the contribution!
Comment on lines
+61
to
+91
| def pytest_collection_modifyitems( | ||
| session: pytest.Session, | ||
| config: TachConfig, | ||
| items: list[pytest.Item], | ||
| ): | ||
| handler = config.tach_handler | ||
| seen: set[Path] = set() | ||
| for item in copy(items): | ||
| if not item.path: | ||
| continue | ||
| if str(item.path) in handler.removed_test_paths: | ||
| handler.num_removed_items += 1 | ||
| items.remove(item) | ||
| continue | ||
| if item.path in seen: | ||
| continue | ||
| paths_to_remove: set[Path] = set() | ||
|
|
||
| if str(item.path) in handler.all_affected_modules: | ||
| # If this test file was changed, | ||
| # then we know we need to rerun it | ||
| seen.add(item.path) | ||
| # First pass: determine which paths should be removed | ||
| for item in items: | ||
| if not item.path or item.path in seen: | ||
| continue | ||
|
|
||
| if handler.should_remove_items(file_path=item.path.resolve()): | ||
| handler.num_removed_items += 1 | ||
| items.remove(item) | ||
| handler.remove_test_path(item.path) | ||
|
|
||
| seen.add(item.path) | ||
|
|
||
| if str(item.path) in handler.removed_test_paths: | ||
| paths_to_remove.add(item.path) | ||
| elif str(item.path) not in handler.all_affected_modules: | ||
| # If this test file was not changed, check if it should be removed | ||
| if handler.should_remove_items(file_path=item.path.resolve()): | ||
| paths_to_remove.add(item.path) | ||
| handler.remove_test_path(item.path) | ||
|
|
||
| # Second pass: remove all items from paths marked for removal | ||
| original_count = len(items) | ||
| items[:] = [ | ||
| item for item in items | ||
| if not item.path or item.path not in paths_to_remove | ||
| ] | ||
| handler.num_removed_items = original_count - len(items) | ||
|
|
Collaborator
There was a problem hiding this comment.
i think we can simplify this by using a pytest_collect_file hook instead:
class HasTachConfig(Protocol):
config: TachConfig
@pytest.hookimpl(wrapper=True)
def pytest_collect_file(
file_path: Path, parent: HasTachConfig
) -> Generator[None, list[pytest.Collector], list[pytest.Collector]]:
handler = parent.config.tach_handler
# skip any paths that already get filtered out by other hook impls
result = yield
if result and handler.should_remove_items(file_path=file_path.resolve()):
handler.num_removed_items += 1
handler.remove_test_path(file_path)
return []
return result
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a bug where only the first test from each unaffected file was skipped, instead of all tests in that file.
The Bug
When running
tach test --base HEAD(no changes), only 1 test per file was being skipped:Root Cause
In
pytest_collection_modifyitems, after processing the first test from a file:seenif item.path in seen: continueThe Fix
Use a two-pass approach:
After Fix
Testing
Tested with a minimal example project: