chore(haste-map): refactor massive class into multiple files#16180
Merged
Conversation
Move copy/copyMap/createEmptyMap to lib/util.ts and extract the ignore matcher logic into lib/buildIgnoreMatcher.ts so ChangeQueue can reuse it without importing from index.ts. HasteMap._ignore now delegates to a stored matcher that is rebuilt when watch mode flips retainAllFiles.
Cache read/persist logic moves to lib/CacheManager.ts. HasteMap.read() and _persist() now delegate to it; setupCachePath creates the instance. The v8 serialize/readFileSync/writeFileSync imports are removed from index.ts.
Worker lifecycle moves to lib/WorkerPool.ts. HasteMap._getWorker and _cleanup now delegate to it; the worker and jest-worker imports are removed from index.ts.
The watchman→node fallback logic moves to crawlers/index.ts. HasteMap._crawl builds the CrawlerOptions and delegates; the individual crawler imports and CrawlerOptions type import are removed from index.ts.
Watcher backend selection, per-root setup, and lifecycle tracking move to watchers/index.ts (WatcherDriver + shouldUseWatchman). HasteMap drops FSEventsWatcher/NodeWatcher/WatchmanWatcher imports, the local Watcher type, _watchers array, _shouldUseWatchman, and isWatchmanInstalledPromise.
Move the change-event accumulation state (promise chain, events queue,
mustCopy flag, emit interval, onChange/emitChange logic) out of the
_watch closure into a dedicated ChangeQueue class. Replace the O(n)
eventsQueue.some() linear scan with a Set<string> keyed on
`${type}:${filePath}:${mtime}` for O(1) dedup.
Move _processFile, _buildHasteMap, and DuplicateError out of the HasteMap god class into FileProcessor. HasteMap keeps thin delegators so call sites are unchanged. DuplicateError moves to FileProcessor to avoid a circular import; index.ts re-exports it to preserve the public API surface.
…n up comments Rebuild _fileProcessor after _watch() mutates retainAllFiles/throwOnModuleCollision so node_modules files get sha1-only processing (not full worker calls) in watch mode. Remove two redundant "what" callback comments from FileProcessor.ts. Remove misleading jest.resetModules() from shouldUseWatchman test.
✅ Deploy Preview for jestjs ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
babel-jest
babel-plugin-jest-hoist
babel-preset-jest
create-jest
@jest/diff-sequences
expect
@jest/expect-utils
jest
jest-changed-files
jest-circus
jest-cli
jest-config
@jest/console
@jest/core
@jest/create-cache-key-function
jest-diff
jest-docblock
jest-each
@jest/environment
jest-environment-jsdom
@jest/environment-jsdom-abstract
jest-environment-node
@jest/expect
@jest/fake-timers
@jest/get-type
@jest/globals
jest-haste-map
jest-jasmine2
jest-leak-detector
jest-matcher-utils
jest-message-util
jest-mock
@jest/pattern
jest-phabricator
jest-regex-util
@jest/reporters
jest-resolve
jest-resolve-dependencies
jest-runner
jest-runtime
@jest/schemas
jest-snapshot
@jest/snapshot-utils
@jest/source-map
@jest/test-result
@jest/test-sequencer
@jest/transform
@jest/types
jest-util
jest-validate
jest-watcher
jest-worker
pretty-format
commit: |
Remove unused WatcherDriver.watcherCount getter. Remove "what" comment from ChangeQueue._pendingEventKeys. Add missing FileProcessor test for getSha1 path (retainAllFiles + node_modules).
There was a problem hiding this comment.
Pull request overview
This PR refactors jest-haste-map by extracting cache, crawl, watch, worker, file-processing, and utility responsibilities out of the main HasteMap class while preserving the public API.
Changes:
- Splits
HasteMapinternals into focused modules underlib/,crawlers/, andwatchers/. - Adds unit tests for the extracted helpers and drivers.
- Replaces watch event duplicate detection with a
Set-based deduplication path.
Reviewed changes
Copilot reviewed 19 out of 19 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
packages/jest-haste-map/src/index.ts |
Rewires HasteMap orchestration to use extracted modules. |
packages/jest-haste-map/src/crawlers/index.ts |
Adds crawler selection and watchman-to-node fallback helper. |
packages/jest-haste-map/src/crawlers/__tests__/index.test.ts |
Tests crawler selection and fallback behavior. |
packages/jest-haste-map/src/lib/CacheManager.ts |
Extracts haste map cache read/write logic. |
packages/jest-haste-map/src/lib/FileProcessor.ts |
Extracts file metadata processing and haste map building. |
packages/jest-haste-map/src/lib/WorkerPool.ts |
Extracts worker lifecycle management. |
packages/jest-haste-map/src/lib/buildIgnoreMatcher.ts |
Extracts ignore predicate construction. |
packages/jest-haste-map/src/lib/util.ts |
Adds shared copy and empty-map helpers. |
packages/jest-haste-map/src/lib/__tests__/CacheManager.test.ts |
Tests cache manager behavior. |
packages/jest-haste-map/src/lib/__tests__/FileProcessor.test.ts |
Tests extracted file processing behavior. |
packages/jest-haste-map/src/lib/__tests__/WorkerPool.test.ts |
Tests worker pool behavior. |
packages/jest-haste-map/src/lib/__tests__/buildIgnoreMatcher.test.ts |
Tests ignore matcher behavior. |
packages/jest-haste-map/src/lib/__tests__/util.test.ts |
Tests shared utility helpers. |
packages/jest-haste-map/src/watchers/ChangeQueue.ts |
Extracts watch change queue processing and Set-based deduplication. |
packages/jest-haste-map/src/watchers/index.ts |
Adds watch backend selection and watcher driver. |
packages/jest-haste-map/src/watchers/__tests__/ChangeQueue.test.ts |
Tests change queue behavior. |
packages/jest-haste-map/src/watchers/__tests__/index.test.ts |
Tests watcher driver and watchman selection behavior. |
packages/jest-haste-map/src/watchers/__tests__/tsconfig.json |
Adds TypeScript test project config for watcher tests. |
Comments suppressed due to low confidence (2)
packages/jest-haste-map/src/crawlers/tests/index.test.ts:74
- This assertion also invokes the watchman-fallback path with the real console, so the expected failure case will emit a warning during a passing test. Use a mocked console for these fallback tests to avoid noisy CI output.
await expect(crawl(crawlerOptions, true, console)).rejects.toThrow(
packages/jest-haste-map/src/crawlers/tests/index.test.ts:83
- This fallback test passes the real console, which causes the warning emitted before the retry failure to appear in normal test output. A mocked console keeps the test quiet and makes the warning behavior explicit.
await expect(crawl(crawlerOptions, true, console)).rejects.toThrow(
Fix import order in ChangeQueue.test.ts (parent paths before local). Pass mocked console to crawl() and FileProcessor in tests that trigger warnings/errors, keeping test output clean.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 19 out of 19 changed files in this pull request and generated 11 comments.
Comments suppressed due to low confidence (10)
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:71
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:95
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:120
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:147
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:165
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:185
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:209
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:233
WorkerPool's constructor requires aworkerPathoption, so this mocked construction is not type-correct and will fail the test typecheck. Include the required option or use a typed mock object instead.
const pool = new MockWorkerPool({maxWorkers: 1});
packages/jest-haste-map/src/watchers/tests/index.test.ts:89
- Untyped
jest.fn()defaults to anunknownreturn type, which makesmockResolvedValue(undefined)invalid under the repository's strict mock typings. Give these close mocks explicit() => Promise<void>types or implementations so this test compiles.
const closeA = jest.fn().mockResolvedValue(undefined);
const closeB = jest.fn().mockResolvedValue(undefined);
packages/jest-haste-map/src/lib/tests/FileProcessor.test.ts:93
- Untyped
jest.fn()defaults to anunknownreturn type, somockRejectedValue(enoent)is invalid with these mock typings unless the mock is typed as returning a Promise. Add an explicit async function type or implementation before configuring the rejection.
worker: jest.fn().mockRejectedValue(enoent),
Add @types/node to the three new __tests__ tsconfigs, export the Callbacks type from ChangeQueue.ts, and rewrite the new test files to use typed jest mock APIs (jest.fn<T>(), jest.mocked(), jest.MockedObject) instead of bare jest.fn() / as-any casts.
When a haste module collision occurred, the code deleted the colliding platform entry from the module map but then checked `Object.keys(moduleMap).length === 1` to decide whether to remove the whole map entry. That check was off-by-one: after deleting one platform, a count of 1 means there is still one valid sibling — only a count of 0 should trigger full deletion.
Three related fixes for watch-mode startup failures: - _createWatcher: name the ready handler so it can be removed on timeout; call watcher.close() after rejecting to avoid leaking the backend handle (B2) - WatcherDriver.start: use Promise.allSettled so all roots are attempted before failing; close successfully-started watchers before throwing an AggregateError when any root fails (B3) - HasteMap._watch: stop the ChangeQueue interval if WatcherDriver.start rejects, so the interval doesn't leak when watch-mode startup fails (B4)
Converts get_mock_name, dependencyExtractor, fast_path, getPlatformExtension, isWatchmanInstalled, and normalizePathSep tests from .js to .ts. Adds @types/node to all four __tests__ tsconfig.json files (extends-first per convention). Also makes DependencyExtractor.extract's filePath and defaultExtract parameters optional since the built-in extractor only uses code.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 29 out of 31 changed files in this pull request and generated 1 comment.
Comments suppressed due to low confidence (2)
packages/jest-haste-map/src/lib/tests/normalizePathSep.test.ts:26
- This mock targets
path, butnormalizePathSepimportsnode:path, so the mock is not applied. On non-Windows hosts the Windows test will keep using the real POSIX separator and fail; mock the samenode:pathspecifier that the module under test imports.
packages/jest-haste-map/src/lib/tests/normalizePathSep.test.ts:14 - This mock uses
path, but the module under test importsnode:path, so this does not replace the dependency. On Windows this test will still load the real Windows path module and the POSIX expectation will fail; mocknode:pathinstead.
This issue also appears on line 23 of the same file.
Comment on lines
5
to
7
| - `[jest-haste-map]` Fix module map entry being deleted when only one platform collides and sibling platforms remain ([#16180](https://github.com/jestjs/jest/pull/16180)) | ||
| - `[expect, jest-message-util, jest-pattern, jest-regex-util, jest-util]` Revert `node:` protocol imports to restore webpack/browser-bundle compatibility ([#16167](https://github.com/jestjs/jest/pull/16167)) | ||
| - `[@jest-environment/jsdom-abstract]` Make `@types/jsdom` a peer dependency ([#16166](https://github.com/jestjs/jest/pull/16166)) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
I am exploring using
@parcel/watcherhttps://www.npmjs.com/package/@parcel/watcher. This first PR splits up the godclass injest-haste-mapto make the future changes easier.Behavior-preserving refactor of
packages/jest-haste-map/src/index.ts. The file was 1157 lines with a singleHasteMapclass owning every concern: option normalization, cache I/O, crawl orchestration, file-processing pipeline, watch orchestration, the change queue, duplicate recovery, and worker management.This PR splits it into focused modules, leaving
index.tsat ~350 lines of orchestration:lib/util.ts—copy,copyMap,createEmptyMaplib/buildIgnoreMatcher.ts— ignore predicate builderlib/CacheManager.ts— v8 serialize/deserialize cache I/Olib/WorkerPool.ts—jest-workerlifecyclecrawlers/index.ts— watchman → node fallback driverwatchers/index.ts—WatcherDriver+shouldUseWatchmanwatchers/ChangeQueue.ts— watch-mode event accumulationlib/FileProcessor.ts—_processFile/buildHasteMap+DuplicateErrorEach extraction is a separate commit with unit tests. The public API (
JestHasteMap,DuplicateError,IHasteMap, all type re-exports) is unchanged.Also includes a small perf fix in
ChangeQueue: the previous O(n) linear scan for duplicate events (eventsQueue.some(...)) is replaced with aSet<string>keyed on${type}:${filePath}:${mtime}, making dedup O(1).Additional fixes included
Bug fix: haste-id module collision deleted surviving platform entries
FileProcessor.tshad an off-by-one: after deleting a colliding platform entry from a module's platform map, it checked=== 1(one entry remaining) before deleting the whole module entry, but should have checked=== 0. This caused sibling platform entries (g,android, etc.) to be incorrectly dropped when only the colliding platform should have been removed.Bug fix: watch-mode startup failure cleanup
Three related issues in
watchers/index.ts:'ready'listener was left registered andwatcher.close()was never called, leaking the backend's resources.Promise.allinWatcherDriver.start()meant that if one root's watcher failed to start, already-started sibling watchers were silently abandoned without being closed. Fixed withPromise.allSettled+ close surviving watchers before re-throwing asAggregateError.index.ts, ifWatcherDriver.start()threw, theChangeQueueinterval that was already started would continue firing indefinitely. Fixed with a try/catch that stops the queue on failure.TypeScript hygiene
lib/__tests__/,crawlers/__tests__/,watchers/__tests__/) now passyarn typecheck:tests. Rewrote mock setup to usejest.fn<typeof SomeFn>(),jest.mocked(), andjest.MockedClass<typeof X>instead ofas any/as unknown as Consolecasts and barejest.fn().mockReturnValue()..jstest files to TypeScript (get_mock_name,dependencyExtractor,fast_path,getPlatformExtension,normalizePathSep,isWatchmanInstalled).Test plan
Green CI. New unit tests added for each extracted module, plus regression tests for the collision fix and watcher cleanup fix.