Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: add test-check workflow - #20935

Closed
DevelopmentCats wants to merge 9 commits into
mainfrom
cat/test-check-action
Closed

feat: add test-check workflow#20935
DevelopmentCats wants to merge 9 commits into
mainfrom
cat/test-check-action

Conversation

@DevelopmentCats

@DevelopmentCats DevelopmentCats commented Nov 25, 2025

Copy link
Copy Markdown
Contributor

This pull request introduces a new GitHub Actions workflow for automated AI-based test coverage analysis on pull requests. The workflow is triggered either by adding the "test-check" label to a PR or via manual dispatch, and creates a Coder Task that reviews PR changes, checks existing tests, and comments with recommendations directly on the PR.

New AI Test Coverage Workflow:

  • Added .github/workflows/test-check.yaml to automate test coverage review using an AI agent, triggered by the "test-check" label or manual workflow dispatch.
  • The workflow builds a detailed prompt for the AI agent, outlining steps to analyze PR changes, assess test coverage for backend (Go), frontend (TypeScript/TSX), E2E, and scale/performance tests, and formats the output as a PR comment.
  • Integrates with the coder/create-task-action to create Coder Tasks, passing the prompt, PR context, and user details securely.
  • Outputs a summary of the task creation, including links and status, to the workflow summary for visibility.

Testing Right Now

@github-actions

github-actions Bot commented Nov 25, 2025

Copy link
Copy Markdown
Contributor

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

✅ No Changes Needed

This PR adds a GitHub Actions workflow configuration file for automated test coverage checking. GitHub Actions workflows are CI/CD infrastructure that don't require unit tests - they're validated through actual execution.


This comment was generated by an AI Agent through Coder Tasks

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

✅ No Changes Needed

This PR adds a new GitHub Actions workflow (.github/workflows/test-check.yaml) that automates test coverage analysis for PRs. This is CI/CD infrastructure code that:

  • Triggers on label addition or manual dispatch
  • Creates Coder Tasks to analyze PR changes
  • Posts automated comments about test coverage

Why no tests are needed:

  • CI/CD workflows are self-validating: The workflow will be tested when it runs in production (triggered by labels or manual dispatch)
  • Similar pattern exists: This follows the same architecture as doc-check.yaml which doesn't have dedicated tests
  • Declarative configuration: GitHub Actions YAML workflows are declarative infrastructure-as-code that GitHub executes
  • Integration testing happens naturally: The workflow's effectiveness will be validated through actual usage

Validation approach:

  • Test by adding the test-check label to a PR or using manual workflow dispatch
  • Verify the Coder Task is created successfully
  • Confirm the task analyzes the PR and posts appropriate comments

This comment was generated by an AI Agent through Coder Tasks

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

✅ No Changes Needed

This PR adds a new GitHub Actions workflow file (.github/workflows/test-check.yaml) which is a CI/CD configuration change. The workflow itself is designed to automate test coverage checks by creating Coder Tasks that analyze PRs.

Reason: CI/CD workflow configuration files don't require unit tests. The workflow will be validated through:

  • GitHub Actions workflow syntax validation
  • Integration testing when the workflow runs on actual PRs with the "test-check" label
  • The workflow's own execution will serve as its functional test

The workflow includes proper error handling, input validation, and clear documentation within the file itself.


This comment was generated by an AI Agent through Coder Tasks

@DevelopmentCats

Copy link
Copy Markdown
Contributor Author

@david-fraley

@david-fraley

Copy link
Copy Markdown
Collaborator

@DevelopmentCats For testing, can you spin up a fake branch based off this one where you make some random change to a coder/coder method/function and see what happens? I'd like to see what the recommendation looks like.

In terms of what to change, maybe try getting the triage workflow to make some backend change, and then add this label to it.

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

❌ Tests Needed

  • coderd/util/strings/strings.go - The new ContainsAny function needs test coverage. This is a utility function with straightforward logic but should include tests for:
    • Empty string input
    • Empty substrings slice
    • No matches found
    • Single match found
    • Multiple matches (should return true on first match)
    • Case sensitivity verification

This comment was generated by an AI Agent through Coder Tasks

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

❌ Tests Needed

  • coderd/util/strings/strings.go - The new ContainsAny function needs test coverage

The PR adds a new exported function ContainsAny(s string, substrings []string) bool that checks if a string contains any of the provided substrings. This is a utility function that should have comprehensive test coverage including:

  • Empty string input
  • Empty substrings list
  • Single substring match
  • Multiple substrings with matches
  • No matches found
  • Case sensitivity verification
  • Edge cases (empty strings in substring list, etc.)

The existing test file at coderd/util/strings/strings_test.go has thorough test coverage for other functions (JoinWithConjunction, Truncate, UISanitize) using table-driven tests. A similar pattern should be followed for ContainsAny.


This comment was generated by an AI Agent through Coder Tasks

@david-fraley david-fraley left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Make sure to remove the func you added before you merge

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

❌ Tests Needed

  • coderd/util/strings/strings_test.go - The EmptyToNil function was modified to treat whitespace-only strings as nil, but there are currently no tests for this function. Tests should cover:
    • Empty string returns nil
    • Whitespace-only strings (spaces, tabs, newlines) return nil
    • Non-empty strings return pointer to the string
    • Strings with leading/trailing whitespace return nil

This comment was generated by an AI Agent through Coder Tasks

Copy link
Copy Markdown
Contributor Author

🧪 Test Coverage Review

⚠️ Tests Need Updates

  • coderd/util/strings/strings_test.go - The test assertions need to be updated to reflect the change from "and" to "or" as the conjunction:
    • Line 16: require.Equal(t, "foo and bar", ...) should be require.Equal(t, "foo or bar", ...)
    • Line 17: require.Equal(t, "foo, bar and baz", ...) should be require.Equal(t, "foo, bar or baz", ...)

This comment was generated by an AI Agent through Coder Tasks

@github-actions github-actions Bot added the stale This issue is like stale bread. label Dec 3, 2025
@github-actions github-actions Bot closed this Dec 7, 2025
@github-actions github-actions Bot removed the stale This issue is like stale bread. label Dec 11, 2025
@github-actions github-actions Bot added the stale This issue is like stale bread. label Dec 19, 2025
@github-actions github-actions Bot closed this Dec 22, 2025
@github-actions
github-actions Bot deleted the cat/test-check-action branch May 26, 2026 00:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread. test-check

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants