feat(test): vitest configuration setup for cli #217
Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. WalkthroughAdds a Vitest-based testing surface: Changes
Sequence Diagram(s)sequenceDiagram
participant Runner as Vitest Runner
participant Setup as test/setup.ts
participant Env as .env.test
participant Mocks as Mock Providers/Services
participant Tests as Unit Tests
Runner->>Setup: load setupFiles
Setup->>Env: read .env.test
Setup-->>Setup: set global.TestConfig (timeout, providers, mockResponses)
Setup->>Mocks: register mocks (AI SDKs, cloud clients, Redis)
Runner->>Tests: run discovered tests
Tests->>Mocks: invoke mocked providers/services
Tests-->>Runner: assertions/results
Runner->>Setup: run teardown hooks
Setup->>Mocks: restore mocks & reset modules
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Potential focus areas:
Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (2 warnings)
✅ Passed checks (1 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: CodeRabbit UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (6)
🧰 Additional context used🧠 Learnings (1)📚 Learning: 2025-08-19T05:02:34.555ZApplied to files:
🪛 dotenv-linter (4.0.0).env.test[warning] 2-2: [UnorderedKey] The MOCK_PROVIDERS key should go before the NODE_ENV key (UnorderedKey) [warning] 3-3: [UnorderedKey] The MOCK_EXTERNAL_SERVICES key should go before the MOCK_PROVIDERS key (UnorderedKey) 🔇 Additional comments (18)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (12)
package.json (3)
59-75: Add UI dependency fortest:ui
vitest --uirequires the UI package. Without it,test:uiwill fail locally/CI.Apply:
"devDependencies": { + "@vitest/ui": "^2.1.9",Also consider pinning the UI version to match
vitest. See version alignment comment below.
202-246: Align Vitest versions to avoid feature skew
vitestis^2.0.0while@vitest/coverage-v8is^2.1.9. Align to the same minor to reduce mismatches.- "@vitest/coverage-v8": "^2.1.9", - "vitest": "^2.0.0", + "@vitest/coverage-v8": "^2.1.9", + "vitest": "^2.1.9",
59-75: Optional: faster local hooksRunning the entire suite on pre-push is slow. Consider a changed-files target for developer ergonomics.
"// Testing (Enhanced Vitest Framework)": "", "test": "vitest run", + "test:changed": "vitest run --changed --logHeapUsage",And in
pre-push, optionally switch topnpm run test:changedfor local pushes; keep fulltest:ciin the release pipeline..env.test (1)
1-3: Use flags to gate mocks in setupGood to keep only non-sensitive values here. Make sure
test/setup.tsactually readsMOCK_PROVIDERSandMOCK_EXTERNAL_SERVICESto conditionally apply mocks; right now mocks are unconditional. See suggested changes in setup.ts.test/setup.ts (5)
8-15: Avoid duplicate mock resets and costly module resetsConfig already sets
clearMocks/mockReset. Repeatingvi.clearAllMocks()/vi.resetAllMocks()each test is redundant;vi.resetModules()each test is expensive and can break hoisted mocks.-beforeEach(() => { - // Reset all mocks before each test - vi.clearAllMocks(); - vi.resetAllMocks(); - - // Reset modules to ensure clean state - vi.resetModules(); -}); +beforeEach(() => { + // Per-test hooks can remain empty; rely on vitest.config.ts for clear/reset. +});Complement this by enabling
restoreMocks: truein vitest.config.ts (see config comment).
22-36: Gate provider/AI mocks behind env flags and broadenaistubsMake mocks conditional and cover common
aiAPIs to prevent accidental breakage.-// Mock AI SDK providers (from implementation document) -vi.mock("ai", () => ({ - stream: vi.fn(), - generate: vi.fn(), - Output: { object: vi.fn() }, -})); +// Mock AI (conditional) +if (process.env.MOCK_PROVIDERS === "true") { + vi.mock("ai", () => { + const fn = () => vi.fn(); + // Provide common API stubs used across the project + return { + // legacy-style + generate: vi.fn(), + stream: vi.fn(), + // [email protected] common helpers + generateText: vi.fn(), + streamText: vi.fn(), + embed: vi.fn(), + }; + }); +}
29-36: Provider mocks: avoid export-name assumptionsExport names differ across
@ai-sdk/*packages. Use a proxy that returns a mock function for any accessed export.-// Mock all AI providers -vi.mock("@ai-sdk/openai", () => ({ openai: vi.fn() })); -vi.mock("@ai-sdk/anthropic", () => ({ anthropic: vi.fn() })); -vi.mock("@ai-sdk/google", () => ({ google: vi.fn() })); -vi.mock("@ai-sdk/google-vertex", () => ({ vertex: vi.fn() })); -vi.mock("@ai-sdk/azure", () => ({ azure: vi.fn() })); -vi.mock("@ai-sdk/mistral", () => ({ mistral: vi.fn() })); +// Mock all AI providers (conditional, export-agnostic) +if (process.env.MOCK_PROVIDERS === "true") { + const anyModule = () => + new Proxy({}, { get: () => vi.fn(), has: () => true }); + vi.mock("@ai-sdk/openai", anyModule); + vi.mock("@ai-sdk/anthropic", anyModule); + vi.mock("@ai-sdk/google", anyModule); + vi.mock("@ai-sdk/google-vertex", anyModule); + vi.mock("@ai-sdk/azure", anyModule); + vi.mock("@ai-sdk/mistral", anyModule); +}Based on learnings.
37-45: Redis mock: add common methods and gate by envMany callers add listeners (
on('error')), duplicate clients, or pipeline. Provide safe no-ops.-// Mock external services -vi.mock("redis", () => ({ - createClient: vi.fn(() => ({ - connect: vi.fn(), - disconnect: vi.fn(), - get: vi.fn(), - set: vi.fn(), - })), -})); +// Mock external services (conditional) +if (process.env.MOCK_EXTERNAL_SERVICES === "true") { + vi.mock("redis", () => ({ + createClient: vi.fn(() => ({ + connect: vi.fn(), + disconnect: vi.fn(), + on: vi.fn(), + duplicate: vi.fn(() => ({ connect: vi.fn(), on: vi.fn() } as any)), + get: vi.fn(), + set: vi.fn(), + del: vi.fn(), + expire: vi.fn(), + multi: vi.fn(() => ({ exec: vi.fn().mockResolvedValue([]) })), + })), + })); +}
48-60: PreferglobalThisfor portabilityUse
globalThisto avoid Node-onlyglobal. Keep the declaration as-is; change the assignment.-global.TestConfig = { +globalThis.TestConfig = { timeout: 30000, providers: ["openai", "anthropic", "google-ai", "bedrock"], mockResponses: true, };test/unit/providers/factory.test.ts (1)
18-24: UseglobalThisfor consistency with setupMinor consistency tweak with the setup’s global assignment.
- it("should have global test config available", () => { - expect(global.TestConfig).toBeDefined(); - expect(global.TestConfig.timeout).toBe(30000); - expect(global.TestConfig.providers).toContain("openai"); - expect(global.TestConfig.providers).toContain("anthropic"); - expect(global.TestConfig.mockResponses).toBe(true); - }); + it("should have global test config available", () => { + expect(globalThis.TestConfig).toBeDefined(); + expect(globalThis.TestConfig.timeout).toBe(30000); + expect(globalThis.TestConfig.providers).toContain("openai"); + expect(globalThis.TestConfig.providers).toContain("anthropic"); + expect(globalThis.TestConfig.mockResponses).toBe(true); + });vitest.config.ts (2)
11-16: EnablerestoreMocksto complement clear/mockResetThis lets you remove manual resets in setup and rely on config.
testTimeout: 30000, // 30s max per test maxConcurrency: 10, // Parallel execution mockReset: true, clearMocks: true, + restoreMocks: true,
49-56: Ensure TS path aliases matchAdd matching
pathsin your tsconfig used by tests (e.g.,tsconfig.jsonor atsconfig.vitest.json) so IDE/type-checkers resolve@,@test,@mockstoo.If not already present, add:
{ "compilerOptions": { "baseUrl": ".", "paths": { "@/*": ["src/*"], "@test/*": ["test/*"], "@mocks/*": ["test/mocks/*"] } } }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
.env.test(1 hunks)package.json(1 hunks)test/setup.ts(1 hunks)test/unit/providers/factory.test.ts(1 hunks)vitest.config.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T05:02:34.555Z
Learnt from: coder-dodo
PR: juspay/neurolink#87
File: test/provider-edge-cases.test.ts:16-21
Timestamp: 2025-08-19T05:02:34.555Z
Learning: When using vi.mock() in Vitest, mocking the same import path that contains the system under test replaces the actual implementation with a mock, even if the physical file has a different extension (.ts vs .js). TypeScript's NodeNext module resolution treats .js imports as references to .ts files.
Applied to files:
test/unit/providers/factory.test.tstest/setup.ts
🪛 dotenv-linter (4.0.0)
.env.test
[warning] 2-2: [UnorderedKey] The MOCK_PROVIDERS key should go before the NODE_ENV key
(UnorderedKey)
[warning] 3-3: [UnorderedKey] The MOCK_EXTERNAL_SERVICES key should go before the MOCK_PROVIDERS key
(UnorderedKey)
🔇 Additional comments (1)
package.json (1)
69-69: I need to better understand the Vitest configuration and reporter support. Let me gather more information:<!-- [scratchpad_update]
[task review_verification 1/2][observations]
- Web search reveals: JUnit reporter IS built-in to Vitest (no plugin needed)
- Supported syntax from official docs:
--reporter=junit --outputFile=test-output.xml(CLI format)- For config file: using outputFile with reporters array is also valid
- Recent Vitest versions support both CLI and config-based output file specification
- JUnit reporter is listed as a standard built-in reporter in Vitest 0.34+
[analysis]
- The initial script failure was due to attempting
--listflag which doesn't exist- The proposed review comment change appears fundamentally correct
- CLI syntax uses
--outputFile=reports/junit.xml(with equals sign)- Documentation shows this works with
--reporter=junit[pending]
- Verify actual vitest version in the repository package.json
- Confirm if repository uses config file or CLI-based approach
- Ensure the proposed path
reports/junit.xmlis appropriate for the project[actions]
- Inspect package.json to find vitest version
- Check if vitest.config.ts/js exists and current configuration
- Verify outputFile path is appropriate for project structure
[scratchpad_end] -->
Now let me check the actual repository setup:<!-- [scratchpad_update]
[task review_verification 1/2][observations]
- Confirmed: JUnit reporter IS built-in to Vitest (no plugin required) from official docs
- Confirmed: CLI syntax for outputFile:
--reporter=junit --outputFile=path/to/file.xml- Multiple sources confirm the reporter is stable and well-supported
- No indication in docs that specific version support is required for junit reporter
- Syntax in the proposed diff looks correct
[analysis]
- Review comment correctly identifies that JUnit output file should be explicitly set
- The proposed diff adds
--outputFile=reports/junit.xmlwhich follows official Vitest documentation- The question about verification has two parts:
- Is JUnit reporter supported? YES (confirmed, built-in)
- Should outputFile be set? YES (best practice for CI/parsers)
[pending]
- Verify exact path
reports/junit.xmlmakes sense for the repository- Confirm if there are other issues with the test:ci script beyond this
[actions]
- Check repository structure for any existing junit output patterns
- Verify no vitest config file exists that would conflict
- Final determination: review comment appears valid and helpful
448fabc to
a308dea
Compare
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Pull Request Overview
This PR establishes a comprehensive Vitest-based testing infrastructure to replace the legacy npx tsx testing approach. It introduces organized test categories (unit, integration, e2e), global mocking for AI providers, path aliases for clean imports, and configurable test scripts with coverage reporting and CI/CD integration.
Key Changes:
- Modern Vitest configuration with performance optimizations (30s timeout, 10 parallel tests, 85% coverage thresholds)
- Global test setup with comprehensive AI provider mocks and environment configuration
- Enhanced test scripts in package.json for various testing scenarios (watch, coverage, CI, debug)
Reviewed Changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| vitest.config.ts | Configures Vitest with performance settings, coverage thresholds, path aliases, and file patterns |
| test/setup.ts | Sets up global test environment with AI provider mocks, cleanup hooks, and test utilities |
| test/unit/providers/factory.test.ts | Initial validation tests ensuring framework configuration works correctly |
| package.json | Adds comprehensive Vitest test scripts while preserving legacy test commands |
| .env.test | Defines test environment variables for isolated testing |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
test/unit/providers/factory.test.ts (1)
33-60: Clarify test status—it's fully functional, not a placeholder.The comment on Line 34 suggests this test awaits future implementation, but the test is already complete and validates mock provider creation, properties, and response generation. Consider updating the comment to reflect the current state.
Apply this diff to clarify:
- // Example test for future provider factory implementation - it("should prepare for provider creation testing", async () => { - // This will be expanded when Ticket #002 (Provider Mocks) is implemented + // Validates mock provider factory pattern + it("should support mock provider creation and invocation", async () => { + // This pattern will be used by the actual ProviderFactory implementationtest/setup.ts (1)
8-15: Consider whethervi.resetModules()is necessary.Line 14 calls
vi.resetModules(), which clears the entire module cache before each test. This can cause unexpected behavior if tests or setup code rely on stateful modules or singleton patterns. Unless you have specific module-state pollution issues, the mock resets (clearAllMocks,resetAllMocks) are typically sufficient.If you encounter module-related issues, you can always add
vi.resetModules()back selectively. Consider this change:beforeEach(() => { // Reset all mocks before each test vi.clearAllMocks(); vi.resetAllMocks(); - - // Reset modules to ensure clean state - vi.resetModules(); });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
.env.test(1 hunks)package.json(1 hunks)test/setup.ts(1 hunks)test/unit/providers/factory.test.ts(1 hunks)vitest.config.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T05:02:34.555Z
Learnt from: coder-dodo
PR: juspay/neurolink#87
File: test/provider-edge-cases.test.ts:16-21
Timestamp: 2025-08-19T05:02:34.555Z
Learning: When using vi.mock() in Vitest, mocking the same import path that contains the system under test replaces the actual implementation with a mock, even if the physical file has a different extension (.ts vs .js). TypeScript's NodeNext module resolution treats .js imports as references to .ts files.
Applied to files:
test/setup.tstest/unit/providers/factory.test.ts
🪛 dotenv-linter (4.0.0)
.env.test
[warning] 2-2: [UnorderedKey] The MOCK_PROVIDERS key should go before the NODE_ENV key
(UnorderedKey)
[warning] 3-3: [UnorderedKey] The MOCK_EXTERNAL_SERVICES key should go before the MOCK_PROVIDERS key
(UnorderedKey)
🔇 Additional comments (8)
.env.test (1)
1-3: LGTM! Test environment configuration is correct.The environment variables properly enable test mode with mocked providers and external services. The static analysis hints about key ordering are pedantic—.env files are typically organized by logical grouping rather than alphabetical order, and the current organization is perfectly acceptable.
test/unit/providers/factory.test.ts (3)
1-12: LGTM! Clean Vitest globals validation.The imports and initial test correctly verify that Vitest globals are properly configured and available.
14-24: LGTM! Environment and global configuration validated correctly.The tests properly verify the test environment setup and global TestConfig presence with expected values.
26-30: LGTM! Basic mocking validation works correctly.test/setup.ts (2)
1-5: LGTM! Environment loading configured correctly.
48-60: LGTM! Global TestConfig properly declared and initialized.The global TypeScript declaration and initialization provide a clean API for tests to access shared configuration. Values align with test expectations in factory.test.ts.
vitest.config.ts (1)
1-51: LGTM! Comprehensive Vitest configuration.The configuration properly sets up:
- Global test environment with Node.js
- Test setup file integration
- Performance optimization (30s timeout, 10 parallel tests)
- Coverage for both SDK and CLI with appropriate thresholds
- Path aliases for cleaner imports
- Test file patterns and exclusions
The previous concern about CLI coverage exclusion has been resolved—both
src/lib/**/*andsrc/cli/**/*are now included.package.json (1)
59-75: LGTM! Well-structured Vitest migration with legacy support.The test script updates demonstrate a thoughtful migration strategy:
- Primary Vitest commands for all test types (unit, integration, e2e)
- Granular test targets (providers, CLI, SDK)
- Coverage and CI integration
- Debug support
- Legacy test command preserved for backward compatibility during transition
- Comparison script to validate migration
This approach minimizes disruption while enabling the new testing infrastructure.
a308dea to
ca6697e
Compare
|
@coderabbitai full review of the changes . |
|
@y-naaz: I'll perform a full review of your Vitest configuration setup. I can see you've implemented a comprehensive testing infrastructure including:
Let me conduct a thorough review of all the changes to ensure everything is properly configured and follows best practices. ✅ Actions performedFull review triggered. |
|
@y-naaz: Thank you for making the updates! I'll perform a comprehensive full review of your Vitest configuration setup, examining: ✅ Configuration Files
✅ Testing Infrastructure
✅ Best Practices
I'll review all the changes you've made and provide detailed feedback on the implementation. ✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
test/setup.ts (2)
30-42: Consider adding mocks for additional providers mentioned in PR description.The PR description mentions support for 9+ major providers including Hugging Face, Ollama, and Mistral AI. Current mocks cover OpenAI, Anthropic, Google (standard + Vertex), Azure, Mistral, and Bedrock. Consider adding mocks for:
@huggingface/inference(Hugging Face)ollama-ai-provider(Ollama)Add these mocks to ensure comprehensive provider coverage:
vi.mock("@ai-sdk/mistral", () => ({ mistral: vi.fn() })); + +// Mock Hugging Face +vi.mock("@huggingface/inference", () => ({ + HfInference: vi.fn(), +})); + +// Mock Ollama +vi.mock("ollama-ai-provider", () => ({ + ollama: vi.fn(), +}));
55-59: Provider naming clarity: "google-ai" vs Google SDK variants.Line 57 uses
"google-ai"in the providers list, but lines 33-34 mock both@ai-sdk/googleand@ai-sdk/google-vertex. Clarify whether "google-ai" represents one or both, or use distinct names like["google", "google-vertex"]to match the SDK structure.Consider this diff for clarity:
global.TestConfig = { timeout: 30000, - providers: ["openai", "anthropic", "google-ai", "bedrock"], + providers: ["openai", "anthropic", "google", "google-vertex", "bedrock"], mockResponses: true, };Then update
test/unit/providers/factory.test.tsline 22 accordingly:- expect(global.TestConfig.providers).toContain("anthropic"); + expect(global.TestConfig.providers).toContain("google");vitest.config.ts (1)
22-35: Coverage configuration uses 85% unified threshold.The configuration now includes both SDK (
src/lib/**/*) and CLI (src/cli/**/*) with a unified 85% threshold. While the PR objectives mention 90% SDK and 85% CLI, this unified approach is pragmatic for initial setup.For future refinement, consider using Vitest's workspace feature to define separate projects with distinct thresholds:
// vitest.workspace.ts (future enhancement) export default [ { extends: './vitest.config.ts', test: { name: 'sdk', include: ['src/lib/**/*.test.ts'], coverage: { thresholds: { global: { branches: 90, ... } } } } }, { extends: './vitest.config.ts', test: { name: 'cli', include: ['src/cli/**/*.test.ts'], coverage: { thresholds: { global: { branches: 85, ... } } } } } ]
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.env.test(1 hunks)package.json(1 hunks)test/setup.ts(1 hunks)test/types/global.ts(1 hunks)test/unit/providers/factory.test.ts(1 hunks)vitest.config.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T05:02:34.555Z
Learnt from: coder-dodo
PR: juspay/neurolink#87
File: test/provider-edge-cases.test.ts:16-21
Timestamp: 2025-08-19T05:02:34.555Z
Learning: When using vi.mock() in Vitest, mocking the same import path that contains the system under test replaces the actual implementation with a mock, even if the physical file has a different extension (.ts vs .js). TypeScript's NodeNext module resolution treats .js imports as references to .ts files.
Applied to files:
vitest.config.tstest/setup.ts
🔇 Additional comments (8)
package.json (2)
73-74: LGTM: Legacy testing support during transition.The preservation of
test:legacyand addition oftest:comparisonprovides a solid migration path, allowing teams to validate that the new Vitest-based tests produce equivalent results to the legacytsx-based test suite.
64-68: Verify test directory structure for scoped test commands.Several test scripts reference directories that may not yet exist:
- Line 66:
test:sdk→test/unit/sdk- Line 67:
test:integration→test/integration- Line 68:
test:e2e→test/e2eEnsure these directories are created (even if empty with a placeholder test) or document that they will be added in follow-up tickets (BZ-45364, BZ-45365).
Run the following script to verify the test directory structure:
.env.test (1)
1-3: LGTM: Clean test environment configuration.The environment variables are appropriately scoped for testing (NODE_ENV=test, mocking enabled). This configuration will be loaded by
test/setup.tsvia dotenv.test/unit/providers/factory.test.ts (2)
1-12: LGTM: Vitest globals validation.This test correctly validates that Vitest globals are available, which is essential for ensuring the test configuration is properly loaded.
18-24: LGTM: Global TestConfig validation.The test appropriately validates the global TestConfig object structure and values, ensuring that the test setup from
test/setup.tsis functioning correctly.test/types/global.ts (1)
6-15: LGTM: Proper global type augmentation.The use of
varin the global declaration is correct for TypeScript global augmentation. This pattern properly types the globalTestConfigobject used across test files.vitest.config.ts (2)
9-19: LGTM: Test configuration aligns with requirements.The test configuration properly sets up:
- Globals enabled for Vitest
- Node environment for testing
- Setup file reference to
test/setup.ts- 30s timeout and 10 parallel tests as per PR requirements
- Mock reset/clear for clean test isolation
48-54: LGTM: Path aliases properly configured.The path aliases (
@,@test,@mocks) use__dirnamefor reliable resolution, addressing the previous review feedback. This ensures consistent import paths across the codebase.
|
@y-naaz: Thank you for implementing the suggested changes! I'll now perform a comprehensive full review of your Vitest configuration setup for PR #217. I'll be examining: ✅ Core Configuration
✅ Test Infrastructure
✅ Quality & Best Practices
I'll provide detailed feedback on all aspects of your implementation to ensure it meets production standards. ✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (2)
test/unit/providers/factory.test.ts (1)
33-60: Provider list inconsistency remains unresolved.Line 35 still defines
mockProviderswith"vertex", butTestConfig.providers(test/setup.ts line 57) uses"bedrock"instead. This inconsistency persists despite the previous review comment being marked as resolved.Apply this diff to align with TestConfig:
- const mockProviders = ["openai", "anthropic", "google-ai", "vertex"]; + const mockProviders = ["openai", "anthropic", "google-ai", "bedrock"];test/setup.ts (1)
1-6: Relative path for .env.test remains unresolved.Line 6 still uses a relative path
".env.test"which was previously flagged for potential path resolution issues when tests run from different directories. The fix was marked as done but hasn't been applied.Apply this diff for reliable path resolution:
import { vi, beforeEach, afterEach } from "vitest"; import * as dotenv from "dotenv"; +import path from "path"; +import { fileURLToPath } from "url"; import "./types/global"; +const __filename = fileURLToPath(import.meta.url); +const __dirname = path.dirname(__filename); + // Load test environment variables -dotenv.config({ path: ".env.test" }); +dotenv.config({ path: path.resolve(__dirname, "../.env.test") });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.env.test(1 hunks)package.json(1 hunks)test/setup.ts(1 hunks)test/types/global.ts(1 hunks)test/unit/providers/factory.test.ts(1 hunks)vitest.config.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T05:02:34.555Z
Learnt from: coder-dodo
PR: juspay/neurolink#87
File: test/provider-edge-cases.test.ts:16-21
Timestamp: 2025-08-19T05:02:34.555Z
Learning: When using vi.mock() in Vitest, mocking the same import path that contains the system under test replaces the actual implementation with a mock, even if the physical file has a different extension (.ts vs .js). TypeScript's NodeNext module resolution treats .js imports as references to .ts files.
Applied to files:
test/setup.tsvitest.config.ts
🪛 dotenv-linter (4.0.0)
.env.test
[warning] 2-2: [UnorderedKey] The MOCK_PROVIDERS key should go before the NODE_ENV key
(UnorderedKey)
[warning] 3-3: [UnorderedKey] The MOCK_EXTERNAL_SERVICES key should go before the MOCK_PROVIDERS key
(UnorderedKey)
🔇 Additional comments (13)
.env.test (1)
1-3: LGTM! Test environment configuration is appropriate.The environment variables correctly configure the test environment with mocking enabled. The static analysis warnings about key ordering are stylistic and can be safely ignored.
test/unit/providers/factory.test.ts (2)
6-24: LGTM! Configuration validation tests are well-structured.The tests appropriately verify Vitest globals, environment setup, and the global TestConfig object. These provide good coverage of the testing infrastructure setup.
26-30: LGTM! Mocking capability verification is appropriate.This test correctly validates that Vitest's mocking functionality is working as expected.
test/setup.ts (4)
8-21: LGTM! Test lifecycle hooks are well-configured.The cleanup strategy using
clearAllMocks,resetAllMocks,resetModulesbefore each test andrestoreAllMocksafter ensures proper test isolation.
23-42: LGTM! Comprehensive AI provider mocking.The mocks cover all major AI providers mentioned in the PR objectives (OpenAI, Anthropic, Google, Vertex, Azure, Mistral, and AWS Bedrock). The simple vi.fn() stubs are appropriate for preventing external API calls during tests.
44-52: LGTM! External service mocking is adequate.The Redis mock provides basic stubbed methods (connect, disconnect, get, set) sufficient for test infrastructure.
54-59: LGTM! Global TestConfig is well-structured.The TestConfig provides appropriate default values for tests. The timeout (30000ms) aligns with vitest.config.ts testTimeout, and the providers list includes the main AI providers being tested.
Note: There's an inconsistency between this providers list and the test expectations in test/unit/providers/factory.test.ts line 35, which has been flagged separately.
test/types/global.ts (1)
1-18: LGTM! Global type definitions are correctly structured.The TestConfigType definition and global augmentation follow TypeScript best practices. The use of
varin the global declaration is correct and necessary, and the eslint-disable comment is appropriate.vitest.config.ts (4)
1-6: LGTM! Proper ES module path resolution.The
__dirnamesetup usingfileURLToPathandimport.meta.urlis the correct pattern for ES modules and addresses reliable path resolution across environments.
8-19: LGTM! Test environment configuration is well-optimized.The configuration appropriately enables Vitest globals, sets up Node environment, and includes performance optimizations (30s timeout, 10 parallel tests) as documented in the PR objectives.
37-44: LGTM! Test discovery patterns are comprehensive.The include/exclude patterns appropriately cover test files in both
srcandtestdirectories while excluding the legacy test file for later migration.
47-54: LGTM! Path aliases are well-configured.The aliases (@, @test, @Mocks) use absolute paths via
__dirname, ensuring reliable import resolution across the codebase as mentioned in PR objectives.package.json (1)
59-74: LGTM! Comprehensive test script suite with good transition support.The Vitest-based test scripts are well-organized, covering unit, integration, e2e, CI/CD, and debugging scenarios. The legacy support commands (test:legacy, test:comparison) facilitate smooth transition from the old testing approach.
Note: Line 66 references
test/unit/sdkwhich may be created in future tickets (BZ-45364, BZ-45365).
ca6697e to
931f6f0
Compare
|
@coderabbitai i have made changes as you suggested can you give full review now |
|
@y-naaz: Thank you for implementing the changes! I'll perform a comprehensive full review of your Vitest configuration setup now. I'll examine all aspects of your implementation including: 🔍 Configuration & Setup
🧪 Testing Infrastructure
✅ Quality & Best Practices
I'll provide detailed feedback on the entire implementation shortly. ✅ Actions performedFull review triggered. |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
test/setup.ts (1)
28-57: Consider mocking Hugging Face and Ollama providers.The setup mocks most AI providers, but
@huggingface/inferenceandollama-ai-provider(both listed in package.json dependencies) are not mocked. If tests will interact with these providers, add corresponding mocks to prevent external API calls during testing.Add mocks for these providers if needed:
vi.mock("@ai-sdk/mistral", () => ({ mistral: vi.fn() })); +vi.mock("@huggingface/inference", () => ({ + HfInference: vi.fn(), +})); +vi.mock("ollama-ai-provider", () => ({ ollama: vi.fn() })); // Mock AWS Bedrock SDK
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.env.test(1 hunks)package.json(1 hunks)test/setup.ts(1 hunks)test/types/global.ts(1 hunks)test/unit/providers/factory.test.ts(1 hunks)vitest.config.ts(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-19T05:02:34.555Z
Learnt from: coder-dodo
PR: juspay/neurolink#87
File: test/provider-edge-cases.test.ts:16-21
Timestamp: 2025-08-19T05:02:34.555Z
Learning: When using vi.mock() in Vitest, mocking the same import path that contains the system under test replaces the actual implementation with a mock, even if the physical file has a different extension (.ts vs .js). TypeScript's NodeNext module resolution treats .js imports as references to .ts files.
Applied to files:
test/setup.tsvitest.config.ts
🪛 dotenv-linter (4.0.0)
.env.test
[warning] 2-2: [UnorderedKey] The MOCK_PROVIDERS key should go before the NODE_ENV key
(UnorderedKey)
[warning] 3-3: [UnorderedKey] The MOCK_EXTERNAL_SERVICES key should go before the MOCK_PROVIDERS key
(UnorderedKey)
🔇 Additional comments (6)
package.json (1)
59-74: Excellent test script organization!The Vitest-based test orchestration is well-structured with clear separation of concerns (unit, integration, e2e), convenient developer commands (watch, ui, debug), and CI integration. The preservation of legacy commands during the transition demonstrates good backward compatibility planning.
.env.test (1)
1-3: LGTM!The test environment variables are clearly defined and appropriate for the test setup. The static analysis warnings about key ordering can be safely ignored—
.envfiles don't have functional ordering requirements, and your current order (general setting first, then specific mocking flags) is logical.test/unit/providers/factory.test.ts (1)
1-61: LGTM!This validation test effectively verifies the Vitest infrastructure setup, including globals, environment configuration, global TestConfig availability, and mocking capabilities. The scaffolded provider creation test (lines 33-60) provides a clear template for future implementation.
test/types/global.ts (1)
1-18: LGTM!The global type definitions follow TypeScript best practices for augmenting the global scope. The use of
varin the global declaration (line 14) is correct and required by TypeScript for global scope augmentation, and the ESLint disable comment appropriately documents this.vitest.config.ts (2)
22-24: Verify JUnit reporter for CI integration.The PR objectives mention "JUnit reporting for CI," but the reporters list only includes
["text", "json", "html", "lcov"]. If CI pipeline integration requires JUnit XML format, add"junit"to the reporters array.Check if your CI system requires JUnit format. If so, apply this diff:
- reporter: ["text", "json", "html", "lcov"], + reporter: ["text", "json", "html", "lcov", "junit"],Note: The
test:ciscript in package.json (line 69) uses--reporter=junit --reporter=verbose, which would override this configuration during CI runs, so this may be optional.
8-67: LGTM!The Vitest configuration is comprehensive and well-structured. Per-path coverage thresholds correctly implement the 90% SDK / 85% CLI requirements, path aliases provide clean imports, and test discovery patterns appropriately cover the codebase while excluding legacy files.
931f6f0 to
615db39
Compare
|
@coderabbitai full review |
✅ Actions performedFull review triggered. |
|
🎉 This PR is included in version 7.53.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Pull Request
Description
This PR implements Ticket BZ-45363: Vitest Configuration Setup - a comprehensive modern testing infrastructure to replace the legacy
npx tsxtesting approach. This establishes the foundation for enterprise-grade testing with performance optimization, coverage tracking, and CI/CD integration.Type of Change
Related Issues
BZ-45364: AI Provider Mock System for CLI
BZ-45365: Test Utilities and Helpers for CLI
Changes Made
Core Infrastructure Files:
vitest.config.ts: Modern Vitest configuration with performance optimizations (30s timeout, 10 parallel tests, coverage thresholds: 90% SDK, 85% CLI)test/setup.ts: Global test setup with comprehensive AI provider mocks and environment configuration.env.test: Test environment variables for isolated testingpackage.json: Enhanced test scripts (test,test:watch,test:coverage,test:ci,test:debug)Test Directory Structure:
test/unit/providers/: Unit tests for AI provider components (<100ms each)test/integration/cli/: CLI integration tests (<5s each)test/integration/mcp/: MCP protocol integration teststest/e2e/workflows/: End-to-end workflow tests (<30s each)test/mocks/: Mock implementations directorytest/utils/: Test utilities and helpers directoryInitial Validation:
test/unit/providers/factory.test.ts: Configuration validation tests ensuring framework works correctlyInfrastructure Improvements:
@,@test,@mocksfor clean importsAI Provider Impact
Component Impact
Testing
Rationale: This testing infrastructure supports all components with organized test categories (unit, integration, e2e) and proper mocking strategies.
Test Environment
Performance Impact
Breaking Changes
No breaking changes
This is a pure addition that maintains 100% backward compatibility. Legacy test command (test:legacy) preserved for gradual migration.
Screenshots/Demo
Primary testing commands
pnpm run test # Execute all tests
pnpm run test:watch # Development watch mode
pnpm run test:coverage # Generate coverage reports
pnpm run test:ci # CI/CD pipeline testing
Targeted testing
pnpm run test:providers # Test AI providers
pnpm run test:cli # Test CLI components
pnpm run test:sdk # Test SDK components
Development & debugging
pnpm run test:debug # Debug mode
pnpm run test:ui # Visual test interface
Checklist
Summary by CodeRabbit