fix(ci): drop continue-on-error from CodeQL analyze job#2178
Merged
imran-siddique merged 1 commit intoMay 12, 2026
Merged
Conversation
`.github/workflows/codeql.yml:29` sets `continue-on-error: true` on the `analyze` job, which means CodeQL failures (autobuild crashes, analyze step errors, SARIF upload failures, init action problems) are recorded as passing checks. The intent on a code-scanning workflow is the opposite: the analysis must succeed or the check should fail so the team can fix the analyzer and not silently drift away from SAST coverage. `strategy.fail-fast: false` already keeps the matrix legs independent — a Python failure doesn't block JavaScript/TypeScript analysis and vice versa. Dropping `continue-on-error` only changes the outcome reported for a leg that genuinely failed; the other leg still runs to completion. Effect after this change — these previously-silent failures will now mark the check as failed: - CodeQL `init` action failure (e.g. ref/SHA pin breakage) - CodeQL `autobuild` action failure (e.g. missing build deps) - CodeQL `analyze` step failure (e.g. queries out of memory) - SARIF upload failure to GitHub Security None of these should be considered passing states; the previous configuration hid real coverage gaps.
🤖 AI Agent: breaking-change-detector — View detailsNo breaking changes detected. |
🤖 AI Agent: security-scanner — View detailsNo security issues found. |
🤖 AI Agent: docs-sync-checker — Docs SyncDocs SyncDocumentation is in sync. |
🤖 AI Agent: test-generator — View detailsTest coverage looks good. No gaps identified. |
🤖 AI Agent: code-reviewer — View detailsTL;DR: 1 blocker, 0 warnings. The change improves security by ensuring CodeQL failures are reported correctly.
Action items: Remove Warnings: No warnings found. Fine as follow-up PRs. |
|
🟡 Contributor Check: MEDIUM
Automated check by AGT Contributor Check. |
PR Review Summary
Verdict: ❌ Changes needed |
MohammadHaroonAbuomar
pushed a commit
to MohammadHaroonAbuomar/agt-acs
that referenced
this pull request
Jun 1, 2026
`.github/workflows/codeql.yml:29` sets `continue-on-error: true` on the `analyze` job, which means CodeQL failures (autobuild crashes, analyze step errors, SARIF upload failures, init action problems) are recorded as passing checks. The intent on a code-scanning workflow is the opposite: the analysis must succeed or the check should fail so the team can fix the analyzer and not silently drift away from SAST coverage. `strategy.fail-fast: false` already keeps the matrix legs independent — a Python failure doesn't block JavaScript/TypeScript analysis and vice versa. Dropping `continue-on-error` only changes the outcome reported for a leg that genuinely failed; the other leg still runs to completion. Effect after this change — these previously-silent failures will now mark the check as failed: - CodeQL `init` action failure (e.g. ref/SHA pin breakage) - CodeQL `autobuild` action failure (e.g. missing build deps) - CodeQL `analyze` step failure (e.g. queries out of memory) - SARIF upload failure to GitHub Security None of these should be considered passing states; the previous configuration hid real coverage gaps.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.github/workflows/codeql.yml:29setscontinue-on-error: trueon theanalyzejob. That means CodeQL failures (autobuild crashes, analyze step errors, SARIF upload failures, init action problems) are recorded as passing checks. The intent on a code-scanning workflow is the opposite: the analysis must succeed or the check should fail, so the team can fix the analyzer and not silently drift away from SAST coverage.strategy.fail-fast: falsealready keeps the matrix legs independent — a Python failure doesn't block JavaScript/TypeScript analysis and vice versa. Droppingcontinue-on-erroronly changes the outcome reported for a leg that genuinely failed; the other leg still runs to completion.(Note:
scorecard.ymldoes not havecontinue-on-error: trueon itsanalyzejob, so this PR is scoped tocodeql.yml. If the reviewer noticed the same shape elsewhere, it's not present today.)Change
Single-line deletion of
continue-on-error: truefrom theanalyzejob header.Effect — these previously-silent failures will now mark the check as failed
initaction failure (e.g. ref/SHA pin breakage)autobuildaction failure (e.g. missing build deps)analyzestep failure (e.g. queries running out of memory)None of these should be considered passing states; the previous configuration hid real coverage gaps.
Verification
actionlintparses cleanly.Surfaced during independent audit conducted by @finnoybu (Ken Tannenbaum, AEGIS Initiative); [LOW, Infrastructure/CI].