Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: prioritize xcresult output over stderr noise in test results#254

Merged
cameroncooke merged 6 commits intogetsentry:mainfrom
czottmann:fix/stderr-warning-classification
Mar 3, 2026
Merged

fix: prioritize xcresult output over stderr noise in test results#254
cameroncooke merged 6 commits intogetsentry:mainfrom
czottmann:fix/stderr-warning-classification

Conversation

@czottmann
Copy link
Contributor

@czottmann czottmann commented Feb 28, 2026

Summary

When xcodebuild tests fail, stderr output (e.g. "multiple matching destinations" warnings) was being tagged as errors and displayed prominently, hiding the actual test failure details from the xcresult bundle.

This PR fixes the response assembly in all three test tool paths (test_macos, test_device, test-common):

  • When tests ran (totalTestCount > 0): xcresult is the authoritative source. Stderr lines are filtered out as redundant noise, and the xcresult summary is placed first in the response.
  • When the build failed (totalTestCount == 0): xcresult is meaningless (empty summary). Stderr is preserved since it contains the actual compilation errors.

No changes to build-utils.ts or stderr capture/classification logic.

Fixes #231

Test plan

  • All 1367 tests pass (1352 passed, 15 skipped)
  • New test: stderr filtered when xcresult has real test data (test_macos, test_device)
  • New test: stderr preserved when xcresult reports 0 tests / build failure (test_macos, test_device)
  • Manual testing: confirmed test failures show xcresult details without stderr wall
  • Manual testing: confirmed build errors (syntax error in Swift file) are still visible when build fails before tests run

@czottmann czottmann force-pushed the fix/stderr-warning-classification branch from c6d1a9d to ecf2288 Compare February 28, 2026 14:16
@czottmann czottmann marked this pull request as draft February 28, 2026 14:47
@czottmann czottmann force-pushed the fix/stderr-warning-classification branch from ecf2288 to 14eaf6e Compare February 28, 2026 15:31
@czottmann czottmann changed the title fix: classify stderr warnings correctly and prioritize xcresult output fix: prioritize xcresult output over stderr noise in test results Feb 28, 2026
@czottmann czottmann marked this pull request as ready for review February 28, 2026 15:33
@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 2, 2026

Open in StackBlitz

npm i https://pkg.pr.new/xcodebuildmcp@254

commit: dba39c1

cameroncooke added a commit that referenced this pull request Mar 2, 2026
…r output

Make xcresult handling consistent across shared, macOS, and device test tools. Parse totalTestCount with strict numeric checks so empty/invalid values cannot suppress build-output fallback behavior.

Filter stderr at line level instead of dropping entire text blocks. This preserves useful status and diagnostics while still removing stderr noise when xcresult summaries are authoritative.

Extract shared stderr content filtering into a common utility to remove duplication and keep behavior aligned across test paths.

Update device tool tests to assert summary-first ordering.

Fixes #254
czottmann and others added 2 commits March 2, 2026 22:49
When xcresult parsing succeeds and tests actually ran (totalTestCount > 0),
stderr lines are redundant noise (e.g. "multiple matching destinations")
and are filtered out. The xcresult summary is placed first in the response.

When xcresult reports 0 tests (build failed before tests could run), the
xcresult is meaningless and stderr is preserved since it contains the
actual compilation errors.

Fixes getsentry#231
…r output

Make xcresult handling consistent across shared, macOS, and device test tools. Parse totalTestCount with strict numeric checks so empty/invalid values cannot suppress build-output fallback behavior.

Filter stderr at line level instead of dropping entire text blocks. This preserves useful status and diagnostics while still removing stderr noise when xcresult summaries are authoritative.

Extract shared stderr content filtering into a common utility to remove duplication and keep behavior aligned across test paths.

Update device tool tests to assert summary-first ordering.

Fixes getsentry#254
@cameroncooke cameroncooke force-pushed the fix/stderr-warning-classification branch from 050c4b7 to 314dc20 Compare March 2, 2026 22:53
…tSummary

Strip orphaned `---` separator lines left behind after filtering [stderr]
lines from Claude Code consolidated output. Also move the XcresultSummary
interface into the shared test-result-content module to eliminate duplication.
…o-warn coercers

Three identical inline coerce functions in cli.ts, yargs-app.ts, and
daemon.ts all mapped 'warning' to 'warn' for backwards compatibility.
Extract a single coerceLogLevel export from logger.ts and reference it
from all three call sites.
…ry boundary

Move consolidation from individual call sites (build-utils, test-common)
into the typed-tool-factory handlers so it applies once at the boundary.
This eliminates scattered consolidation calls and simplifies logic functions.

Supporting changes:
- Cache isRunningUnderClaudeCode() result since it runs on every tool call
- Accept optional EnvironmentDetector in consolidation for testability
- Inject CommandExecutor/FileSystemExecutor into test-common for DI
- Simplify filterStderrContent now that separator cleanup is unnecessary
- Add unit tests for consolidation logic and factory wiring
- Reduce verbose type guards in test_macos formatTestSummary using
  direct casts and nullish coalescing
- Remove unused JSDoc typedef for TestSummary in test_macos
- Consolidate enforceInstallPolicy guard conditions into single check
- Simplify init report agentsGuidance construction
- Remove stale comments in test_device
Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

@cameroncooke cameroncooke merged commit 3611437 into getsentry:main Mar 3, 2026
6 checks passed
cameroncooke added a commit that referenced this pull request Mar 3, 2026
…r output

Make xcresult handling consistent across shared, macOS, and device test tools. Parse totalTestCount with strict numeric checks so empty/invalid values cannot suppress build-output fallback behavior.

Filter stderr at line level instead of dropping entire text blocks. This preserves useful status and diagnostics while still removing stderr noise when xcresult summaries are authoritative.

Extract shared stderr content filtering into a common utility to remove duplication and keep behavior aligned across test paths.

Update device tool tests to assert summary-first ordering.

Fixes #254
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: "multiple matching destinations" warning reported as error instead of failing tests

2 participants