feat: add --fail-hook-affected-tests option to report skipped tests as failed #5519

akshah123 · 2025-11-01T15:05:41Z

When before() or beforeEach() hooks fail, Mocha currently only reports the hook failure and silently skips affected tests. This makes it difficult to track test coverage and understand the full impact of hook failures.

This change introduces a new CLI option --fail-hook-affected-tests that, when enabled, reports all tests affected by hook failures as failed instead of silently skipping them.

Changes:

Added --fail-hook-affected-tests boolean CLI option
Implemented failAffectedTests() method in Runner to fail all affected tests
Updated hook() method to call failAffectedTests when hooks fail
Added failHookAffectedTests option to Mocha class
Added integration tests for both before() and beforeEach() hook failures
All affected tests now receive descriptive error messages indicating which hook caused them to be skipped

Fixes #4392

PR Checklist

Addresses an existing open issue: fixes 🚀 Feature: Failures in before() and beforeAll() should cause all impacted tests to be reported as failed #4392
That issue was marked as status: accepting prs
Steps in CONTRIBUTING.md were taken

Overview

linux-foundation-easycla · 2025-11-01T15:05:48Z

✅ login: akshah123 / name: Ankit Shah (1feba53, 9fe8493, c171cf1)
❌ - login: @claude / name: Claude . The commit (6d3d64e) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please submit a support request ticket.

mark-wiemer · 2025-11-01T17:18:14Z

Hmm, I think we can ignore the CLA issue as the non-signer is AI :D

mark-wiemer · 2025-11-01T17:22:40Z

I'll review this in more detail sometime in the next ~7 days :)

codecov · 2025-11-08T22:03:08Z

Codecov Report

❌ Patch coverage is 94.00000% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.69%. Comparing base (2a0bce0) to head (9fe8493).
⚠️ Report is 16 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/mocha.js	33.33%	2 Missing ⚠️
lib/runner.js	97.87%	1 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5519   +/-   ##
=======================================
  Coverage   93.69%   93.69%           
=======================================
  Files          57       57           
  Lines        4391     4440   +49     
  Branches      850      861   +11     
=======================================
+ Hits         4114     4160   +46     
- Misses        277      280    +3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mark-wiemer

Overall looks OK, just requesting some renames for clarity. I'll check the coverage and make sure I can repro this one locally as it's a big change. Fortunately we should be able to include this in 12.0.0-beta-1 :)

lib/runner.js

test/integration/fixtures/hooks/before-each-hook-error-with-fail-affected.fixture.js

test/integration/fixtures/hooks/before-hook-error-with-fail-affected.fixture.js

test/integration/fixtures/hooks/before-each-hook-error-with-fail-affected.fixture.js

test/integration/fixtures/hooks/before-hook-error-with-fail-affected.fixture.js

test/integration/hook-err.spec.js

…s failed When `before()` or `beforeEach()` hooks fail, Mocha currently only reports the hook failure and silently skips affected tests. This makes it difficult to track test coverage and understand the full impact of hook failures. This change introduces a new CLI option `--fail-hook-affected-tests` that, when enabled, reports all tests affected by hook failures as failed instead of silently skipping them. Changes: - Added --fail-hook-affected-tests boolean CLI option - Implemented failAffectedTests() method in Runner to fail all affected tests - Updated hook() method to call failAffectedTests when hooks fail - Added failHookAffectedTests option to Mocha class - Added integration tests for both before() and beforeEach() hook failures - All affected tests now receive descriptive error messages indicating which hook caused them to be skipped Fixes mochajs#4392

- Extract error message creation into createHookSkipError() helper function to eliminate duplication across three locations - Simplify recursive forEach call in failAffectedTests() - Update test fixture and suite descriptions to use backticks for clarity (e.g., 'error in `before` hook')

mark-wiemer · 2025-11-21T03:37:10Z

Took me a minute to remember I needed to actively add the --fail-hook-affected-tests flag to see the new behavior :D was able to repro locally and the code does look good. Going to wait for @JoshuaKGoldberg to review, hopefully early next week.

JoshuaKGoldberg

Generally looking good - the approach is solid and respects existing patterns. Great! 🙌

Requesting changes on the [Bug] around throwing non-Error values. Otherwise we're looking pretty good to merge soon.

@mark-wiemer shall we back-port this to v11?

lib/mocha.js

lib/runner.js

@JoshuaKGoldberg

- Fix JSDoc placement for failHookAffectedTests and failZero - Handle non-Error thrown values in createHookSkipError - Add tests for null, undefined, and other non-Error throws Addresses review comments from @JoshuaKGoldberg: - Separated JSDoc for failHookAffectedTests from failZero docs - Added proper handling for falsy and non-Error exceptions using createInvalidExceptionError and thrown2Error patterns - Added comprehensive test coverage for edge cases 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <[email protected]>

akshah123 · 2025-12-19T00:25:14Z

@JoshuaKGoldberg Thanks for the thorough review! I've addressed both of your comments in the latest commit (c171cf1):

Re: [Docs] JSDoc placement - Fixed! The JSDoc now correctly documents failHookAffectedTests and has been separated from failZero's documentation. Each function now has its own properly positioned JSDoc comment.

Re: [Bug] Non-Error handling - Great catch! The createHookSkipError function now properly handles:

Falsy values (null/undefined) using createInvalidExceptionError
Non-Error objects (strings, numbers, etc.) using thrown2Error

This follows the existing patterns from runner.js:1112 and runner.js:537. I've also added comprehensive test coverage for these edge cases with new fixture files testing null, undefined, string, and number throws in both before and beforeEach hooks.

All tests are passing, including the 2 new integration tests specifically for non-Error handling. ✅

JoshuaKGoldberg

LGTM. One little touchup but we can always apply this before merging. Thanks! ⭐

lib/runner.js

Co-authored-by: Josh Goldberg ✨ <[email protected]>

mark-wiemer · 2025-12-19T07:59:14Z

@akshah123 looks like your latest commit broke some stuff, please review the failed checks.

@JoshuaKGoldberg happy to port this back to v11, created #5580

mark-wiemer changed the title ~~feat: add --fail-hook-affected-tests option to report skipped tests a…~~ feat: add --fail-hook-affected-tests option to report skipped tests as failed Nov 1, 2025

mark-wiemer mentioned this pull request Nov 1, 2025

🛠 Repo: EasyCLA fails PRs co-authored by AI #5521

Open

5 tasks

mark-wiemer self-requested a review November 1, 2025 17:22

akshah123 force-pushed the claude/fix-issue-4392-011CUfwRYBHqzwYtxQzmsCdN branch from 5dee260 to 37acdb2 Compare November 3, 2025 14:22

mark-wiemer requested changes Nov 8, 2025

View reviewed changes

claude and others added 2 commits November 9, 2025 11:24

akshah123 force-pushed the claude/fix-issue-4392-011CUfwRYBHqzwYtxQzmsCdN branch from ade4687 to 9fe8493 Compare November 9, 2025 16:24

akshah123 requested a review from mark-wiemer November 9, 2025 16:24

mark-wiemer approved these changes Nov 21, 2025

View reviewed changes

mark-wiemer added this to Maintenance Reboot Nov 22, 2025

mark-wiemer self-assigned this Nov 22, 2025

This was referenced Nov 22, 2025

Report skipped tests upon beforeEach hook failure #4233

Closed

Report skipped tests upon beforeAll hook failure #4221

Closed

mark-wiemer requested a review from JoshuaKGoldberg November 22, 2025 06:27

mark-wiemer added this to the v12.x milestone Nov 22, 2025

JoshuaKGoldberg requested changes Dec 18, 2025

View reviewed changes

lib/mocha.js Show resolved Hide resolved

lib/runner.js Show resolved Hide resolved

JoshuaKGoldberg added the status: waiting for author waiting on response from OP or other posters - more information needed label Dec 18, 2025

akshah123 requested a review from JoshuaKGoldberg December 19, 2025 00:26

JoshuaKGoldberg approved these changes Dec 19, 2025

View reviewed changes

lib/runner.js Outdated Show resolved Hide resolved

Update lib/runner.js

1feba53

Co-authored-by: Josh Goldberg ✨ <[email protected]>

Uh oh!

feat: add --fail-hook-affected-tests option to report skipped tests as failed #5519

Are you sure you want to change the base?

feat: add --fail-hook-affected-tests option to report skipped tests as failed #5519

Uh oh!

Conversation

akshah123 commented Nov 1, 2025 • edited by mark-wiemer Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Checklist

Overview

Uh oh!

linux-foundation-easycla bot commented Nov 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mark-wiemer commented Nov 1, 2025

Uh oh!

mark-wiemer commented Nov 1, 2025

Uh oh!

codecov bot commented Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mark-wiemer left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mark-wiemer commented Nov 21, 2025

Uh oh!

JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

akshah123 commented Dec 19, 2025

Uh oh!

JoshuaKGoldberg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mark-wiemer commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

akshah123 commented Nov 1, 2025 •

edited by mark-wiemer

Loading

linux-foundation-easycla bot commented Nov 1, 2025 •

edited

Loading

codecov bot commented Nov 8, 2025 •

edited

Loading