Fix testcontainers cleanup #2023
Conversation
|
WalkthroughThis set of changes introduces enhanced logging, diagnostics, and stricter TypeScript checks for test container lifecycle management. The core test container utilities are refactored to provide detailed, structured logs and metadata for setup and cleanup phases, with new modules for logging and Docker diagnostics. Test files in the run engine package are updated to properly await asynchronous engine shutdowns, ensuring correct resource cleanup. The GitHub Actions workflow is updated to configure Docker networking and disable IPv6 before running unit tests. Additionally, dependency versions are updated and stricter TypeScript options are enabled. Changes
Sequence Diagram(s)sequenceDiagram
participant TestRunner
participant TestContainerUtils
participant Docker
participant Logger
TestRunner->>TestContainerUtils: Setup resource (with TaskContext)
TestContainerUtils->>Logger: logSetup(resource, metadata)
TestContainerUtils->>Docker: Start container / network
Docker-->>TestContainerUtils: Container/Network started
TestContainerUtils->>Logger: logSetup complete
TestRunner->>TestContainerUtils: Use resource
TestContainerUtils->>Logger: log usage metadata
TestRunner->>TestContainerUtils: Cleanup resource
TestContainerUtils->>Logger: logCleanup(resource, promise, metadata)
TestContainerUtils->>Docker: Stop container / network
Docker-->>TestContainerUtils: Container/Network stopped
TestContainerUtils->>Logger: logCleanup complete (with diagnostics if needed)
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
✅ Files skipped from review due to trivial changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (7)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
internal-packages/testcontainers/src/index.ts (1)
73-73: Fix empty object pattern.There's an empty object pattern
{}in the function parameters which is unnecessary.-const network = async ({}, use: Use<StartedNetwork>) => { +const network = async (_: {}, use: Use<StartedNetwork>) => {or
-const network = async ({}, use: Use<StartedNetwork>) => { +const network = async (_, use: Use<StartedNetwork>) => {🧰 Tools
🪛 Biome (1.9.4)
[error] 73-73: Unexpected empty object pattern.
(lint/correctness/noEmptyPattern)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (2)
internal-packages/testcontainers/package.json(1 hunks)internal-packages/testcontainers/src/index.ts(5 hunks)
🧰 Additional context used
🪛 Biome (1.9.4)
internal-packages/testcontainers/src/index.ts
[error] 73-73: Unexpected empty object pattern.
(lint/correctness/noEmptyPattern)
⏰ Context from checks skipped due to timeout of 90000ms (7)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: units / 🧪 Unit Tests
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: Analyze (javascript-typescript)
🔇 Additional comments (7)
internal-packages/testcontainers/package.json (1)
13-14: Version bump for testcontainers dependencies looks good.The update of testcontainers-related packages from
^10.13.1to^10.25.0aligns with the PR's goal of fixing testcontainers cleanup. This minor version update likely includes improvements to container management and cleanup mechanisms.Also applies to: 16-16
internal-packages/testcontainers/src/index.ts (6)
34-71: Well-designed cleanup logging mechanism.This new implementation significantly improves the observability of container cleanup operations. The
logCleanupfunction provides:
- Detailed timing information (start, end, duration)
- Cleanup order tracking
- Concurrency detection
- Structured error handling
- JSON-formatted logs for easier parsing
This will be valuable for diagnosing test container issues, especially in CI environments.
78-82: Good refactoring of network cleanup.The network cleanup now uses the central logging mechanism, providing consistency with other resource cleanups.
95-97: Appropriate timeout increase for container stopping.Increasing the container stop timeout from the default to 30 seconds is a good practice. This gives containers more time to shut down gracefully, potentially preventing issues in subsequent tests caused by lingering containers.
119-121: Good application of logCleanup to the Prisma client.Consistently applying the same logging pattern to database disconnection will help track database-related cleanup issues.
140-142: Consistent timeout for Redis container.The 30-second timeout for Redis container is consistent with other containers, ensuring a standardized approach to cleanup.
194-196: Electric container cleanup matches other containers.The electric container cleanup now follows the same pattern as other containers with the standardized 30-second timeout.
There was a problem hiding this comment.
Actionable comments posted: 0
🔭 Outside diff range comments (1)
.github/workflows/testcontainers.yml (1)
1-78: 💡 Verification agent🧩 Analysis chain
Well-structured parallel testing workflow with Docker configuration
This new workflow efficiently runs tests in parallel with 20 matrix runs, which should significantly improve test execution time. The Docker address pool configuration matches the one added to the unit-tests workflow, ensuring consistent container networking.
A few recommendations:
- Add GitHub token permissions to limit the scope of the workflow
- Fix the trailing spaces in the YAML file for better code quality
Add a permissions block to limit token scope:
name: "🐳 Testcontainers" on: workflow_call: workflow_dispatch: push: +permissions: + contents: read jobs: unitTests: name: "🧪 Unit Tests (run ${{ matrix.run }})"And fix trailing spaces on lines 72 and 75:
SESSION_SECRET: "secret" MAGIC_LINK_SECRET: "secret" ENCRYPTION_KEY: "secret" - + - name: 🧪 Run Package Unit Tests run: pnpm run test:packages - + - name: 🧪 Run Internal Unit Tests
🌐 Web query:
Best practices for GitHub Actions token permissions💡 Result:
Best Practices for GitHub Actions Token Permissions
GitHub Actions provides an automated token (
GITHUB_TOKEN) that grants permissions to access resources within your repository. Managing these permissions properly is crucial for maintaining security in your CI/CD workflows. Here are the best practices for handling GitHub Actions token permissions:Follow the Principle of Least Privilege
The most fundamental best practice is to restrict
GITHUB_TOKENpermissions to only what's absolutely necessary for each workflow:
- Set read-only permissions by default at the repository or organization level[4][6]
- Explicitly define permissions needed for specific jobs or workflows[5]
- Avoid using the default permissions which might be overly permissive[7]
For example, if your workflow only needs to read repository contents, you should explicitly set:
permissions: contents: readConfigure Default Settings at Organization Level
For enterprises and organizations, the most effective approach to implement least privilege is to:
- Configure "Read repository contents and package permissions" as the default workflow permissions at the organization level[4]
- This ensures all GitHub Actions workflows have only read permissions for contents and packages by default
- Individual workflows can then request additional permissions as needed[6]
Use Job-Specific Permissions
When different jobs within a workflow need different permission levels, set them at the job level rather than the workflow level:
jobs: read-only-job: permissions: contents: read runs-on: ubuntu-latest steps: # Steps that only need to read content deployment-job: permissions: contents: write deployments: write runs-on: ubuntu-latest steps: # Steps that need to write content or create deploymentsThis approach ensures each job has only the permissions it specifically requires[5].
Be Cautious with Third-Party Actions
When using actions created by third parties, it's particularly important to restrict permissions:
- Limit the
GITHUB_TOKENpermissions when using third-party actions[5]- Consider using a more restrictive set of permissions for workflows that include third-party code[7]
For example:
permissions: actions: none checks: read contents: readAvoid Using Personal Access Tokens When Possible
The
GITHUB_TOKENis preferable to personal access tokens (PATs) in most cases because:
- It's automatically created and expired for each job[1]
- It has built-in scope limitations to the repository where the workflow runs[2]
- It's easier to audit and manage across an organization[4]
When you do need a PAT, create fine-grained tokens with limited repository access and specific permissions[8].
Implement Additional Security Controls
Beyond permission settings, consider these additional security controls:
- Use the StepSecurity Harden-Runner to restrict egress network traffic and implement runtime security controls[4]
- Monitor workflow runs for unexpected permission usage or suspicious activities
- Set expiration for any tokens that you create manually[8]
Monitor and Audit Token Usage
Regularly review:
- The permissions being used in your workflows
- The build logs of workflow runs (look at the 'Set up job' step to check
GITHUB_TOKENpermissions)[4]- Any changes to workflows that might modify permission scopes
By implementing these best practices, you can significantly reduce the security risks associated with GitHub Actions workflows while ensuring they have the necessary permissions to function properly.
Citations:
- 1: https://docs.github.com/actions/security-guides/automatic-token-authentication
- 2: https://docs.github.com/en/actions/security-for-github-actions/security-guides/security-hardening-for-github-actions
- 3: https://docs.github.com/en/actions/writing-workflows/choosing-what-your-workflow-does/controlling-permissions-for-github_token
- 4: https://www.stepsecurity.io/blog/github-actions-security-best-practices
- 5: https://dev.to/pwd9000/fgjgghjgh-19ka
- 6: https://www.stepsecurity.io/blog/github-token-how-it-works-and-how-to-secure-automatic-github-action-tokens
- 7: https://blog.gitguardian.com/github-actions-security-cheat-sheet/
- 8: https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens
🏁 Script executed:
#!/bin/bash # Check for trailing whitespace in the workflow file rg -n '\s+$' .github/workflows/testcontainers.yml || trueLength of output: 71
Add least-privileged GitHub token permissions and remove trailing whitespace
To follow the principle of least privilege, explicitly scope
GITHUB_TOKENto only read repository contents, and clean up trailing spaces in the workflow file:• File:
.github/workflows/testcontainers.yml
– Insert apermissionsblock immediately after theon:section:name: "🐳 Testcontainers" on: workflow_call: workflow_dispatch: push: +permissions: + contents: read jobs: unitTests: …– Remove the two trailing-space lines detected (lines 72 and 75):
ENCRYPTION_KEY: "secret" - + - name: 🧪 Run Package Unit Tests run: pnpm run test:packages - + - name: 🧪 Run Internal Unit Tests🧰 Tools
🪛 Checkov (3.2.334)
[MEDIUM] 67-68: Basic Auth Credentials
(CKV_SECRET_4)
🪛 YAMLlint (1.35.1)
[error] 72-72: trailing spaces
(trailing-spaces)
[error] 75-75: trailing spaces
(trailing-spaces)
🪛 GitHub Check: CodeQL
[warning] 10-77: Workflow does not contain permissions
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}
🧹 Nitpick comments (1)
internal-packages/testcontainers/src/index.ts (1)
73-73: Fix empty object patternThe empty object pattern
{}should be replaced with a named parameter or removed for clarity.-const network = async ({}, use: Use<StartedNetwork>) => { +const network = async (_: {}, use: Use<StartedNetwork>) => {🧰 Tools
🪛 Biome (1.9.4)
[error] 73-73: Unexpected empty object pattern.
(lint/correctness/noEmptyPattern)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
.github/workflows/testcontainers.yml(1 hunks).github/workflows/unit-tests.yml(1 hunks)internal-packages/testcontainers/src/index.ts(5 hunks)
🧰 Additional context used
🪛 Checkov (3.2.334)
.github/workflows/testcontainers.yml
[MEDIUM] 67-68: Basic Auth Credentials
(CKV_SECRET_4)
🪛 YAMLlint (1.35.1)
.github/workflows/testcontainers.yml
[error] 72-72: trailing spaces
(trailing-spaces)
[error] 75-75: trailing spaces
(trailing-spaces)
🪛 GitHub Check: CodeQL
.github/workflows/testcontainers.yml
[warning] 10-77: Workflow does not contain permissions
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}
🪛 Biome (1.9.4)
internal-packages/testcontainers/src/index.ts
[error] 73-73: Unexpected empty object pattern.
(lint/correctness/noEmptyPattern)
⏰ Context from checks skipped due to timeout of 90000ms (27)
- GitHub Check: 🧪 Unit Tests (run 20)
- GitHub Check: 🧪 Unit Tests (run 19)
- GitHub Check: 🧪 Unit Tests (run 18)
- GitHub Check: 🧪 Unit Tests (run 17)
- GitHub Check: 🧪 Unit Tests (run 16)
- GitHub Check: 🧪 Unit Tests (run 15)
- GitHub Check: 🧪 Unit Tests (run 14)
- GitHub Check: 🧪 Unit Tests (run 13)
- GitHub Check: 🧪 Unit Tests (run 12)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: 🧪 Unit Tests (run 11)
- GitHub Check: 🧪 Unit Tests (run 10)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: 🧪 Unit Tests (run 9)
- GitHub Check: 🧪 Unit Tests (run 8)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: 🧪 Unit Tests (run 7)
- GitHub Check: units / 🧪 Unit Tests
- GitHub Check: typecheck / typecheck
- GitHub Check: 🧪 Unit Tests (run 6)
- GitHub Check: 🧪 Unit Tests (run 5)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: 🧪 Unit Tests (run 4)
- GitHub Check: 🧪 Unit Tests (run 3)
- GitHub Check: 🧪 Unit Tests (run 2)
- GitHub Check: 🧪 Unit Tests (run 1)
🔇 Additional comments (5)
.github/workflows/unit-tests.yml (1)
11-27: Docker configuration enhances testing reliabilityAdding this Docker address pool configuration is a good way to prevent network address conflicts when running containers during tests. This approach helps ensure consistent test environments and reduces potential flaky tests due to networking issues.
Consider adding a brief comment explaining why this configuration is necessary for better context for future developers.
internal-packages/testcontainers/src/index.ts (4)
34-71: Great improvement to resource cleanup logging and error handlingThe new
logCleanupfunction is an excellent addition that addresses several important aspects of container cleanup:
- Structured JSON logging provides consistent format for analyzing cleanup patterns
- Tracking cleanup order and parallelism helps understand resource disposal sequence
- Error handling ensures cleanup failures are logged but don't interrupt the process
- Timing information helps identify slow cleanups that might need optimization
The implementation is thorough and well-documented.
79-79: Improved network cleanup with centralized error handlingThe network cleanup now uses the
logCleanupfunction, which provides consistent error handling and logging. This is an improvement over the previous implementation.
93-93: Increased container stop timeout improves cleanup reliabilityIncreasing the container stop timeout from 10 to 30 seconds is a good change that gives containers more time to shut down gracefully. This helps prevent potential issues with lingering containers and improves test stability.
Also applies to: 134-134, 186-186
115-115: Enhanced Prisma client cleanupUsing
logCleanupfor Prisma client disconnection provides consistent error handling and logging, aligning with the overall cleanup strategy improvements in this PR.
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (1)
.github/workflows/testcontainers.yml (1)
3-7:⚠️ Potential issueAdd explicit permissions to restrict GITHUB_TOKEN
The workflow doesn't specify permissions for the GITHUB_TOKEN, which is a security best practice to limit permissions to only what's needed.
Add a permissions block to restrict token access:
on: workflow_call: workflow_dispatch: push: +permissions: + contents: read jobs:
🧹 Nitpick comments (3)
.github/workflows/testcontainers.yml (2)
64-72: Consider using environment secrets for database credentialsWhile hardcoded credentials are acceptable for local test databases, consider using GitHub secrets for consistency with your other secrets.
Also, remove the trailing space at line 72.
- name: 🧪 Run Webapp Unit Tests run: pnpm run test:webapp env: DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres DIRECT_URL: postgresql://postgres:postgres@localhost:5432/postgres SESSION_SECRET: "secret" MAGIC_LINK_SECRET: "secret" ENCRYPTION_KEY: "secret" - +🧰 Tools
🪛 Checkov (3.2.334)
[MEDIUM] 67-68: Basic Auth Credentials
(CKV_SECRET_4)
🪛 YAMLlint (1.35.1)
[error] 72-72: trailing spaces
(trailing-spaces)
73-75: Remove trailing spaceRemove the trailing space at line 75.
- name: 🧪 Run Package Unit Tests run: pnpm run test:packages - +🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 75-75: trailing spaces
(trailing-spaces)
internal-packages/testcontainers/src/index.ts (1)
221-221: Fix the empty object pattern in function parameterThere's an empty object pattern in the network function parameter that should be properly typed.
-const network = async ({}, use: Use<StartedNetwork>) => { +const network = async ({}: Record<string, never>, use: Use<StartedNetwork>) => {🧰 Tools
🪛 Biome (1.9.4)
[error] 221-221: Unexpected empty object pattern.
(lint/correctness/noEmptyPattern)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (17)
.github/workflows/testcontainers.yml(1 hunks).github/workflows/unit-tests.yml(1 hunks)internal-packages/run-engine/src/engine/tests/attemptFailures.test.ts(6 hunks)internal-packages/run-engine/src/engine/tests/batchTrigger.test.ts(1 hunks)internal-packages/run-engine/src/engine/tests/batchTriggerAndWait.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/cancelling.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/checkpoints.test.ts(1 hunks)internal-packages/run-engine/src/engine/tests/delays.test.ts(4 hunks)internal-packages/run-engine/src/engine/tests/dequeuing.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/pendingVersion.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/priority.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/trigger.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/triggerAndWait.test.ts(2 hunks)internal-packages/run-engine/src/engine/tests/ttl.test.ts(1 hunks)internal-packages/run-engine/src/engine/tests/waitpoints.test.ts(8 hunks)internal-packages/testcontainers/package.json(1 hunks)internal-packages/testcontainers/src/index.ts(6 hunks)
✅ Files skipped from review due to trivial changes (1)
- internal-packages/run-engine/src/engine/tests/attemptFailures.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
- internal-packages/testcontainers/package.json
- .github/workflows/unit-tests.yml
🧰 Additional context used
🧬 Code Graph Analysis (3)
internal-packages/run-engine/src/engine/tests/batchTrigger.test.ts (1)
apps/webapp/app/v3/runEngine.server.ts (1)
engine(9-9)
internal-packages/run-engine/src/engine/tests/batchTriggerAndWait.test.ts (1)
apps/webapp/app/v3/runEngine.server.ts (1)
engine(9-9)
internal-packages/run-engine/src/engine/tests/checkpoints.test.ts (1)
apps/webapp/app/v3/runEngine.server.ts (1)
engine(9-9)
🪛 Biome (1.9.4)
internal-packages/testcontainers/src/index.ts
[error] 221-221: Unexpected empty object pattern.
(lint/correctness/noEmptyPattern)
🪛 Checkov (3.2.334)
.github/workflows/testcontainers.yml
[MEDIUM] 67-68: Basic Auth Credentials
(CKV_SECRET_4)
🪛 YAMLlint (1.35.1)
.github/workflows/testcontainers.yml
[error] 72-72: trailing spaces
(trailing-spaces)
[error] 75-75: trailing spaces
(trailing-spaces)
🪛 GitHub Check: CodeQL
.github/workflows/testcontainers.yml
[warning] 10-77: Workflow does not contain permissions
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}
⏰ Context from checks skipped due to timeout of 90000ms (13)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: units / 🧪 Unit Tests
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: 🧪 Unit Tests (run 10)
- GitHub Check: 🧪 Unit Tests (run 9)
- GitHub Check: 🧪 Unit Tests (run 8)
- GitHub Check: 🧪 Unit Tests (run 7)
- GitHub Check: 🧪 Unit Tests (run 6)
- GitHub Check: 🧪 Unit Tests (run 5)
- GitHub Check: 🧪 Unit Tests (run 4)
- GitHub Check: 🧪 Unit Tests (run 3)
- GitHub Check: 🧪 Unit Tests (run 2)
- GitHub Check: 🧪 Unit Tests (run 1)
🔇 Additional comments (26)
internal-packages/run-engine/src/engine/tests/ttl.test.ts (1)
105-105: Added await to ensure proper asynchronous cleanupThe change properly awaits the asynchronous
engine.quit()call, ensuring that the engine's resources are fully cleaned up before the test completes. This prevents potential resource leaks and ensures consistent test behavior.internal-packages/run-engine/src/engine/tests/batchTrigger.test.ts (1)
180-180: Added await to ensure proper asynchronous cleanupThe change properly awaits the asynchronous
engine.quit()call, ensuring that the engine's resources are fully cleaned up before the test completes. This prevents potential resource leaks and ensures consistent test behavior.internal-packages/run-engine/src/engine/tests/cancelling.test.ts (2)
223-223: Added await to ensure proper asynchronous cleanupThe change properly awaits the asynchronous
engine.quit()call, ensuring that the engine's resources are fully cleaned up before the test completes. This prevents potential resource leaks and ensures consistent test behavior.
324-324: Added await to ensure proper asynchronous cleanupThe change properly awaits the asynchronous
engine.quit()call, ensuring that the engine's resources are fully cleaned up before the test completes. This prevents potential resource leaks and ensures consistent test behavior.internal-packages/run-engine/src/engine/tests/batchTriggerAndWait.test.ts (2)
355-355: Added await to ensure proper asynchronous cleanupThe change properly awaits the asynchronous
engine.quit()call, ensuring that the engine's resources are fully cleaned up before the test completes. This prevents potential resource leaks and ensures consistent test behavior.
573-573: Added await to ensure proper asynchronous cleanupThe change properly awaits the asynchronous
engine.quit()call, ensuring that the engine's resources are fully cleaned up before the test completes. This prevents potential resource leaks and ensures consistent test behavior.internal-packages/run-engine/src/engine/tests/delays.test.ts (1)
89-89: Great improvement to properly await engine.quit() callsAdding
awaitto these asynchronous cleanup calls ensures that the engine shutdown process completes before the test finishes. This prevents potential resource leaks and improves test reliability by ensuring proper cleanup between tests.Also applies to: 186-186, 290-290, 401-401
internal-packages/run-engine/src/engine/tests/pendingVersion.test.ts (1)
161-161: Correctly awaiting asynchronous cleanupAdding
awaitto these engine.quit() calls ensures proper cleanup after complex tests involving multiple runs, queues, and database state transitions. This change prevents potential resource leaks and test interference.Also applies to: 322-322
internal-packages/run-engine/src/engine/tests/checkpoints.test.ts (1)
1378-1378: Completed async cleanup patternAdding
awaithere completes the pattern of properly awaiting asynchronous cleanup calls in this file. This ensures that all resources created during this checkpoint test are properly cleaned up before the test terminates.internal-packages/run-engine/src/engine/tests/triggerAndWait.test.ts (1)
192-192: Properly awaiting asynchronous cleanup in parent-child run testsAdding
awaitto these cleanup calls ensures that all resources created during complex parent-child run tests are fully cleaned up before test completion. This is particularly important when testing coordinated runs with waitpoints, as incomplete cleanup could leave dangling resources or state.Also applies to: 448-448
internal-packages/run-engine/src/engine/tests/priority.test.ts (2)
106-106: Properly await the asynchronous engine.quit() callThe addition of
awaitbeforeengine.quit()ensures proper asynchronous cleanup of resources before the test case finishes. This prevents potential resource leaks and race conditions that could lead to flaky tests.
200-200: Properly await the asynchronous engine.quit() callThe addition of
awaitbeforeengine.quit()ensures proper asynchronous cleanup of resources before the test case finishes. This is consistent with the fix in the previous test case in this file.internal-packages/run-engine/src/engine/tests/dequeuing.test.ts (2)
80-80: Properly await the asynchronous engine.quit() callAdding
awaitbeforeengine.quit()ensures proper asynchronous cleanup of resources before the test case finishes. This prevents potential resource leaks and test pollution that could affect subsequent tests.
172-172: Properly await the asynchronous engine.quit() callAdding
awaitbeforeengine.quit()ensures proper asynchronous cleanup of resources before the test case finishes. This is consistent with the fix in the previous test case in this file.internal-packages/run-engine/src/engine/tests/waitpoints.test.ts (1)
124-124: Properly await all asynchronous engine.quit() calls across multiple test casesAll instances of
engine.quit()have been updated toawait engine.quit()across the test cases in this file. This ensures proper asynchronous cleanup of resources before each test case finishes, preventing resource leaks and potential test interference.This change is particularly important in tests with waitpoints and complex concurrency patterns, as proper cleanup prevents race conditions and zombie processes that could affect test reliability.
Also applies to: 264-264, 403-403, 519-519, 667-667, 817-817, 969-969, 1129-1129
internal-packages/run-engine/src/engine/tests/trigger.test.ts (3)
201-201: Properly await the asynchronous engine.quit() callAdding
awaitbeforeengine.quit()ensures proper asynchronous cleanup of resources before the test case finishes. This is important for the successful run test case to cleanly release all resources.
328-328: Properly await the asynchronous engine.quit() callAdding
awaitbeforeengine.quit()ensures proper asynchronous cleanup of resources before the test case finishes. This is consistent with the fix in the successful run test case and ensures proper cleanup even when tests end with failures.
1-331: Great job improving resource cleanup across testsThese changes collectively address a systematic issue with resource cleanup in tests. By properly awaiting the asynchronous
engine.quit()calls, you've improved test reliability and prevented potential resource leaks and race conditions that could lead to flaky tests.This change aligns well with the broader improvements in the PR for standardizing and enhancing cleanup operations with the new
logCleanuphelper function mentioned in the PR summary..github/workflows/testcontainers.yml (3)
1-16: Good choice of parallelization strategyThe workflow setup with a matrix of 10 parallel runs and fail-fast disabled is a good approach for testcontainer tests. This allows isolation between test runs and helps identify flaky tests by running them multiple times in parallel.
17-34: Good Docker configuration to avoid network conflictsConfiguring Docker's default address pools is an excellent practice for container-heavy CI workflows. This prevents IP address conflicts when running multiple containers in parallel across the test matrix.
I see you've configured:
- A 172.17.0.0/12 pool with size 20
- A 192.168.0.0/16 pool with size 24
This should provide sufficient IP space for your containerized tests.
51-57: Nice usage of DockerHub authenticationUsing DockerHub authentication is a good practice to avoid rate limits when pulling images, especially in CI environments with multiple parallel runs.
internal-packages/testcontainers/src/index.ts (5)
8-9: Good additions for better diagnostics and CI detectionAdding
tinyexecfor shell command execution andstd-envfor CI environment detection are good choices for the enhanced logging system.
36-37: Well-designed global state for tracking cleanup operationsThese global counters are a good approach for tracking cleanup sequence and concurrency:
cleanupOrdermaintains the sequence of resource cleanupsactiveCleanupstracks concurrent cleanup operationsThis will be valuable for diagnosing resource cleanup issues.
39-91: Excellent implementation of structured cleanup loggingThe
logCleanupfunction is a well-implemented enhancement:
- It standardizes cleanup logging across different resource types
- It tracks important diagnostics: timing, errors, order, and concurrency
- It conditionally collects Docker diagnostics when cleanup takes too long or fails
- It only logs in CI environments to avoid overhead in local development
- It uses structured JSON logging for easy parsing and analysis
This will significantly improve debugging of container cleanup issues.
93-219: Comprehensive Docker diagnostics implementationThe Docker diagnostic functions provide valuable insights for debugging container-related issues:
- They collect information about Docker networks, containers, and their relationships
- They have proper error handling to prevent diagnostics from throwing errors
- They return well-structured data
- They're only triggered when there's a problem, minimizing overhead
These diagnostics will be especially helpful when troubleshooting network or container cleanup issues.
227-227: Good standardization of resource cleanup with increased timeoutsAll cleanup operations now use the
logCleanuphelper with consistent 30-second timeouts for containers. This is a significant improvement:
- The increased timeout (from 10 to 30 seconds) gives containers more time to shut down gracefully
- Standardizing cleanup through
logCleanupensures consistent error handling and diagnostics- The diagnostic data will help identify resource cleanup issues
This change addresses likely cleanup races that can occur in parallel test environments.
Also applies to: 241-241, 263-263, 282-282, 334-334
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
internal-packages/testcontainers/src/index.ts (2)
73-79: Consider adding a configurable threshold for Docker diagnosticsThe function triggers Docker diagnostics if cleanup takes longer than 5 seconds. Consider making this threshold configurable via an environment variable to adjust sensitivity based on the testing environment.
- if (error || durationMs > 5000 || env.DOCKER_DIAGNOSTICS) { + const diagnosticsThresholdMs = Number(env.DOCKER_DIAGNOSTICS_THRESHOLD_MS) || 5000; + if (error || durationMs > diagnosticsThresholdMs || env.DOCKER_DIAGNOSTICS) {
133-176: Consider implementing rate limiting for Docker commands in parallel test environmentsWhen many tests run in parallel, each calling Docker diagnostic commands, there's potential for performance issues or rate limiting from the Docker daemon. Consider implementing a simple rate limiting mechanism or caching for Docker diagnostics.
// Add this somewhere at the top of the file let lastDockerDiagnosticsTime = 0; let cachedDiagnostics: DockerDiagnostics | null = null; const DOCKER_DIAGNOSTICS_CACHE_MS = 1000; // 1 second cache // Modify getDockerDiagnostics to use caching async function getDockerDiagnostics(): Promise<DockerDiagnostics> { const now = Date.now(); // Return cached diagnostics if recent enough if (cachedDiagnostics && now - lastDockerDiagnosticsTime < DOCKER_DIAGNOSTICS_CACHE_MS) { return cachedDiagnostics; } const [containers, networks, networkAttachments, containerNetworks] = await Promise.all([ getDockerContainers(), getDockerNetworks(), getDockerNetworkAttachments(), getDockerContainerNetworks(), ]); cachedDiagnostics = { containers, networks, containerNetworks, networkAttachments, }; lastDockerDiagnosticsTime = now; return cachedDiagnostics; }Also applies to: 182-223
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
internal-packages/testcontainers/src/index.ts(4 hunks)internal-packages/testcontainers/src/utils.ts(1 hunks)
🧰 Additional context used
🧬 Code Graph Analysis (1)
internal-packages/testcontainers/src/index.ts (2)
apps/webapp/app/env.server.ts (1)
env(727-727)internal-packages/testcontainers/src/utils.ts (3)
createPostgresContainer(9-42)createRedisContainer(44-75)createElectricContainer(107-129)
⏰ Context from checks skipped due to timeout of 90000ms (17)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - pnpm)
- GitHub Check: e2e / 🧪 CLI v3 tests (windows-latest - npm)
- GitHub Check: 🧪 Unit Tests (run 10)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - pnpm)
- GitHub Check: 🧪 Unit Tests (run 9)
- GitHub Check: e2e / 🧪 CLI v3 tests (ubuntu-latest - npm)
- GitHub Check: typecheck / typecheck
- GitHub Check: 🧪 Unit Tests (run 8)
- GitHub Check: units / 🧪 Unit Tests
- GitHub Check: 🧪 Unit Tests (run 7)
- GitHub Check: 🧪 Unit Tests (run 6)
- GitHub Check: 🧪 Unit Tests (run 5)
- GitHub Check: 🧪 Unit Tests (run 4)
- GitHub Check: 🧪 Unit Tests (run 3)
- GitHub Check: Analyze (javascript-typescript)
- GitHub Check: 🧪 Unit Tests (run 2)
- GitHub Check: 🧪 Unit Tests (run 1)
🔇 Additional comments (13)
internal-packages/testcontainers/src/utils.ts (2)
90-94: Good enhancement to container metadata collectionAdding structured container metadata collection improves observability for Redis connection issues. The metadata includes essential container details like ID, name, and network names, which will make debugging easier.
97-97: Improved error handling for Redis connection issuesPreviously, errors were silently caught by the event handler without logging. Now, both the error and container metadata are logged, which will provide better visibility into Redis connection failures during tests.
internal-packages/testcontainers/src/index.ts (11)
6-9: Appropriate imports for new functionalityAdded imports support the new logging and diagnostics functionality. Using
std-envfor environment detection is a good practice for conditional logging.
36-37: Well-designed counter implementation for tracking cleanup and setup operationsThe global counters
cleanupOrder,activeCleanups, andsetupOrderenable tracking of the sequence and concurrency of cleanup and setup operations. This is essential for diagnosing issues in parallel test execution environments.Also applies to: 330-331
39-97: Excellent implementation of centralized cleanup loggingThe
logCleanupfunction provides a standardized way to handle resource cleanup with detailed logging. It captures important metrics:
- Timing information (start, end, duration)
- Cleanup order and parallelism tracking
- Error handling and reporting
- Conditional Docker diagnostics based on failure or slow cleanup
The CI-only logging ensures logs don't clutter local development environments.
99-105: Simple and effective string parsing utilitiesThese utility functions have clear purposes and implementations for parsing Docker command output.
107-246: Comprehensive Docker diagnostics implementationThe Docker diagnostic functions provide detailed information about Docker resources, which is essential for debugging container issues. The functions handle errors gracefully and always return some information, even when parts of the diagnostics gathering fail.
The Docker command format strings are well-constructed to extract relevant information.
248-272: Proper integration of task context and cleanup logging for network resourcesThe network fixture now receives test context information and implements the new logging pattern. The network stop is properly wrapped with
logCleanupto ensure consistent error handling and logging.
274-302: Good enhancement to postgres container managementThe postgres container fixture now includes detailed logging and uses a 30-second timeout for container stop operations, which is more reliable than the default timeout. The metadata collection is comprehensive and useful for debugging.
332-348: Well-implemented setup logging functionThe
logSetupfunction complementslogCleanupby providing similar structured logging for resource setup operations. The CI-only logging avoids noise in local environments.
301-301: Appropriate increase in container stop timeoutIncreasing the container stop timeout from the default 10 seconds to 30 seconds is a good practice. This allows more time for containers to properly shut down, reducing the likelihood of cleanup errors, especially in CI environments where resources might be constrained.
Also applies to: 382-382, 451-451
350-383: Consistent pattern applied to Redis container managementThe Redis container fixture follows the same pattern as other resources, with detailed logging and proper cleanup timeout. This consistency makes the code more maintainable.
421-452: Proper enhancement to Electric container managementThe Electric container fixture now includes TaskContext, detailed logging, and proper cleanup timeout, consistent with other container resources.
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/testcontainers.yml (1)
8-16:⚠️ Potential issueAdd permissions block to the workflow
The workflow doesn't specify permissions for the GITHUB_TOKEN, which could be a security concern. GitHub's principle of least privilege suggests limiting permissions to only what's needed.
Add a permissions block at the workflow level:
name: "🐳 Testcontainers" on: workflow_call: workflow_dispatch: push: +permissions: + contents: read jobs: unitTests: name: "🧪 Unit Tests (run ${{ matrix.run }})" runs-on: ubuntu-latest strategy: fail-fast: false matrix: run: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
🧹 Nitpick comments (4)
.github/workflows/testcontainers.yml (4)
17-22: Consider explaining why IPv6 is disabledThe workflow disables IPv6, but it's not clear why this is necessary. Adding a comment explaining the reason would improve maintainability.
- name: 🔧 Disable IPv6 + # Disabling IPv6 to prevent potential networking issues with Docker containers run: | sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1 sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1 sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1
80-80: Remove trailing whitespaceThere's trailing whitespace at the end of this line.
- ENCRYPTION_KEY: "secret" - + ENCRYPTION_KEY: "secret"🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 80-80: trailing spaces
(trailing-spaces)
83-83: Remove trailing whitespaceThere's trailing whitespace at the end of this line.
- run: pnpm run test:packages - + run: pnpm run test:packages🧰 Tools
🪛 YAMLlint (1.35.1)
[error] 83-83: trailing spaces
(trailing-spaces)
72-85: Consider adding timeout for test runsThe test steps don't have timeout limits specified. Consider adding timeouts to prevent jobs from running indefinitely if something gets stuck, especially with container-based tests.
- name: 🧪 Run Webapp Unit Tests + timeout-minutes: 15 run: pnpm run test:webapp env: DATABASE_URL: postgresql://postgres:postgres@localhost:5432/postgres DIRECT_URL: postgresql://postgres:postgres@localhost:5432/postgres SESSION_SECRET: "secret" MAGIC_LINK_SECRET: "secret" ENCRYPTION_KEY: "secret" - name: 🧪 Run Package Unit Tests + timeout-minutes: 15 run: pnpm run test:packages - name: 🧪 Run Internal Unit Tests + timeout-minutes: 15 run: pnpm run test:internal🧰 Tools
🪛 Checkov (3.2.334)
[MEDIUM] 75-76: Basic Auth Credentials
(CKV_SECRET_4)
🪛 YAMLlint (1.35.1)
[error] 80-80: trailing spaces
(trailing-spaces)
[error] 83-83: trailing spaces
(trailing-spaces)
🪛 GitHub Check: CodeQL
[warning] 10-85: Workflow does not contain permissions
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/testcontainers.yml(1 hunks)internal-packages/testcontainers/src/utils.ts(3 hunks)
🧰 Additional context used
🪛 Checkov (3.2.334)
.github/workflows/testcontainers.yml
[MEDIUM] 75-76: Basic Auth Credentials
(CKV_SECRET_4)
🪛 YAMLlint (1.35.1)
.github/workflows/testcontainers.yml
[error] 80-80: trailing spaces
(trailing-spaces)
[error] 83-83: trailing spaces
(trailing-spaces)
🪛 GitHub Check: CodeQL
.github/workflows/testcontainers.yml
[warning] 10-85: Workflow does not contain permissions
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {{contents: read}}
⏰ Context from checks skipped due to timeout of 90000ms (11)
- GitHub Check: units / 🧪 Unit Tests
- GitHub Check: 🧪 Unit Tests (run 10)
- GitHub Check: 🧪 Unit Tests (run 9)
- GitHub Check: 🧪 Unit Tests (run 8)
- GitHub Check: 🧪 Unit Tests (run 7)
- GitHub Check: 🧪 Unit Tests (run 6)
- GitHub Check: 🧪 Unit Tests (run 5)
- GitHub Check: 🧪 Unit Tests (run 4)
- GitHub Check: 🧪 Unit Tests (run 3)
- GitHub Check: 🧪 Unit Tests (run 2)
- GitHub Check: 🧪 Unit Tests (run 1)
🔇 Additional comments (6)
internal-packages/testcontainers/src/utils.ts (4)
72-77: Improved error handling for Redis connection verificationThe new implementation properly handles errors from
verifyRedisConnectionusingtryCatch, ensuring that containers are cleaned up with an appropriate timeout when verification fails. This is a good improvement to the resource management.
97-101: Good addition of container metadata for debuggingCollecting container metadata (ID, name, network names) provides valuable context for troubleshooting Redis connection issues. This will help diagnose problems when containers fail to start properly.
103-109: Improved error handling with conditional loggingThe error handler now includes debug-mode logging with container metadata, which will be extremely helpful for troubleshooting. The comment explaining that you're not throwing immediately is also helpful for understanding the control flow.
113-119: Better error propagation for ping failuresGood improvement to the error handling by providing additional context when Redis ping fails. The conditional debug logging with container metadata will significantly help with diagnostics.
.github/workflows/testcontainers.yml (2)
23-41: Good Docker configuration for preventing network conflictsThe Docker address pool configuration is a good practice to prevent network conflicts when running multiple containers in parallel. This aligns well with the matrix strategy for test parallelization.
59-64: Secure handling of DockerHub credentialsGood use of GitHub secrets for DockerHub authentication to avoid rate limits. This is a security best practice.
Turns out there were a few issues:
container.getPort()may return a random port if it differed from the IPv4 mapping.Bit of a rabbit hole, but we now also have:
Summary by CodeRabbit
New Features
Bug Fixes
Chores