fix(replay): carry optional-skip warnings to every human surface and hint iOS viewport refusals - #2571
Open
thymikee wants to merge 3 commits into
Open
fix(replay): carry optional-skip warnings to every human surface and hint iOS viewport refusals#2571thymikee wants to merge 3 commits into
thymikee wants to merge 3 commits into
Conversation
…hint viewport refusals An `optional: true` Maestro step that skipped stayed a warning that the terminal never printed once a later step failed: the accumulated channel rode only on the success response, so the repro run in #2560 reported a bare failure with no trace of the skip. The engine now emits warnings on the failure event, the daemon copies them onto the failure response's error details, and the failed test result carries the same array the passed result already had; `replay` prints them through its facet formatter and the test reporter adds a Warnings section that renders whether tests pass or fail. The skip warning also appends the skipped failure's hint, so guidance that lived only in error details reaches the terminal. A regular iOS presentation refusal now carries a hint pointing at `snapshot --raw`, and when the tree came from a system surface the hint names that surface and the error details carry its registry identity, so an `optional` capture miss caused by the acquisition boundary stops looking like a selector problem.
A failing `replay` still printed no trace of the run's accumulated warnings: the human error renderer read message, hint, and divergence but never the error-level warnings channel, so the pass/fail asymmetry survived on that surface. It now prints each warning after the error. Round-two corrections: warnings on the divergence response are scrubbed against replay vars like message and hint already are; a failed attempt copies warnings only when non-empty, matching the passing branch; the optional-skip hint append is removed because no producer of a maestro test failure emits a hint — capture-error lineage stays with #2450 — along with the CHANGELOG claim it rested on; the failed-result copy gains its own module test, and JUnit covers a failed test's warning lines.
…ays present An empty array and an absent key said the same thing two ways, so every consumer needed both guards. The failed outcome now always carries the warnings array, like the passed outcome already did, and the result builder narrows the attempt once instead of per field. This also removes the clone the shared candidate-view formatter had grown between the CLI and MCP error renderers — the MCP error text gained the same warnings lines the CLI stderr prints.
|
Size Report
Startup median (7 runs, lower is better):
|
Member
Author
|
No code findings at fb07951: skipped-step warnings reach the failure response and human reporters, with replay-variable scrubbing preserved. Checks are green. The Apple presentation hint is also a changed device-facing path, so fixture evidence alone leaves a validation gap: please record a live simulator refusal on a confirmed SafariViewService sheet and verify the hint and systemSurface details. The underlying sheet-interaction fix remains outside this PR. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refs #2560 (capture-routing/lineage half stays with #2450/#2485).
An
optional: trueMaestro step that skipped left a warning that vanished from human output the moment a later step failed — the repro printed a bareREPLAY_DIVERGENCEwith no trace of the 7s skip. Warnings now travel with the run whether it passes or fails:warningsarray the passing result already had; the default reporter'sWarnings:section, JUnit lines, failingreplaystderr, and MCP error text all render them.snapshot --raw; when the tree came from a system surface (e.g.SafariViewServicesheet) the hint names the surface and error details carry its registry identity, so a capture-boundary miss stops looking like a selector bug.37 files; the deep test-vs-replay routing diff is not in this PR.
Validation
Tested at
fb07951;pnpm check:affected --runpassed on that head (format, lint, typecheck, unit lanes incl. apple-runner, output-economy, maestro-compat, fallow, layering, wire/replay-compat, command-docs). New tests are red without their respective hunks. No device run: changes are response/reporter plumbing only; the reporter-path fix is fixture-proven, the live SafariViewService scenario remains reproducible only on the reporter's host.