Fix appended system-message test event subscription race - #2636
Draft
devm33 wants to merge 1 commit into
Draft
Conversation
Subscribe before sending and reuse the existing RPC event fence to reproduce early ephemeral idle delivery in the E2E test's actual response wait. Co-authored-by: Copilot <[email protected]> Copilot-Session: faf9a106-bb3d-40e7-bf7b-9fce99d945a7
Contributor
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The focused test-only changes correctly eliminate the event-subscription race and preserve existing assertions.
Review tier: Balanced
Findings: None
What changed in this PR
Fixes an E2E race by subscribing for completion events before sending the prompt.
Changes:
- Replaces split send/wait calls with
SendAndWaitAsync. - Adds deterministic regression coverage for early idle delivery.
| File | Description |
|---|---|
dotnet/test/E2E/SessionE2ETests.cs |
Uses race-safe response waiting. |
dotnet/test/Unit/ClientSessionLifetimeTests.cs |
Tests assistant/idle events arriving before the send reply. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fix the response wait in
SessionE2ETests.Should_Create_A_Session_With_Appended_SystemMessage_Config. It subscribed throughGetFinalAssistantMessageAsynconly after awaitingSendAsync. A completed turn can deliver its ephemeralsession.idlefirst; durable history then contains the assistant message but cannot supply the missing idle notification.Use
SendAndWaitAsyncto subscribe before sending. Preserve the original 120-second E2E timeout, prompt, response assertions (GitHubandHave a nice day!), and captured system-message assertions. No SDK or runtime production code changes.The deterministic regression drives the E2E test's actual response-wait scenario. It reuses the fake RPC server added in #2635, delivers assistant/idle events, and drains a later event before replying to
session.send. It also verifies that durable history contains the assistant but no idle event.Failure evidence
Observed during validation of github/copilot-agent-runtime#20258: first-attempt CAPI run
34677727987, job103512036461, runtime head4a585d08bf, SDK commita6f2e56acb5e04af56241146668fe71291304266. The appended-system-message case timed out atTestHelper.cs:77/85, called fromSessionE2ETests.cs:58; that backend finished with 892 passes, one failure, and four skips.The original CI event trace was not uploaded, so its exact interleaving is unknown. The following controlled reproductions establish a live test race with the same failure, not retrospective proof of that CI interleaving.
Validation
SendAsync's return until actual idle delivery: failed withTimeout waiting for assistant messageafter the unchanged 120-second wait. The probe observed one durable assistant message and zero durable idle events.ClientSessionLifetimeTestsandSessionE2ETestsfiles passed: 164 tests onnet8.0, no skips.dotnet format test/GitHub.Copilot.SDK.Test.csproj --no-restore --verify-no-changespassed.Local replay validation used the freshly built runtime at dependency-integration commit
44613db704; toolchain was .NET SDK 10.0.401 with the required .NET 8 runtime. No fixture changes are included.Ten independent
.NET SDK Testsworkflow runs were dispatched on exact head34f8036f87aa4e186804066cea00d4f529b78190, each verified as attempt 1:34680184778,34680186084,34680187444,34680188695,34680189919,34680190926,34680191987,34680193191,34680194253,34680195312. These are new workflow runs, not reruns of failed checks.The population is incomplete: six workflows succeeded; four hit the macOS default/CAPI shard 1 job limit. GitHub labels runs
34680184778,34680187444,34680188695, and34680189919as cancelled, but each affected job's annotation explicitly reports the 20-minute execution limit. They are not manual cancellations or passes, and a timeout alone does not establish an infrastructure failure.The first timed-out job (
103517319544) completed setup but never recorded completion of the test step. Its job log returns 404, no diagnostic artifact was retained, and the full workflow archive omits that shard. No individual test can yet be attributed from that evidence.Both the repaired appended-system-message E2E and its early-idle regression are verified 10/10 passing in the Ubuntu/default/CAPI/full jobs, using original runner-native result lines, exact first-attempt metadata, and hashed logs. No test failures were found in those selected logs; incidental cache-reservation messages were kept separate. These target-level passes do not clear the four macOS job timeouts or establish ten successful whole workflows. No failed jobs have been rerun.
Exact-head Copilot review
5185720603recommended approval with no findings and no inline threads. This is automated review clearance, not a formal human approval or completion of the ten-success CI requirement.Generated by Copilot