copilot: emit router decision to restricted telemetry full scores#312230
Open
copilot: emit router decision to restricted telemetry full scores#312230
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a restricted/enhanced GitHub telemetry event for Auto Mode router decisions so router quality signals can be analyzed within the restricted telemetry pipeline.
Changes:
- Emit a new
automode.routerDecisionRestrictedenhanced telemetry event alongside the existingautomode.routerDecisionMSFT event. - Include additional router output fields (candidate list, chosen model, hydra/binary scores, shortfall, per-class scores) in the restricted event’s payload and GDPR annotation.
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/platform/endpoint/node/routerDecisionFetcher.ts | Adds a second telemetry emission to the enhanced/restricted pipeline with additional router score details and metadata. |
Copilot's findings
- Files reviewed: 1/1 changed files
- Comments generated: 4
bhavyaus
reviewed
Apr 23, 2026
bhavyaus
approved these changes
Apr 23, 2026
sbatten
approved these changes
Apr 23, 2026
18c2832 to
9cd119c
Compare
Add a sendEnhancedGHTelemetryEvent call (automode.routerDecisionRestricted) alongside the existing unrestricted routerDecision event. The restricted event includes: - hydraScores: JSON-encoded 4-dim capability scores - binaryScores: JSON-encoded binary classifier scores - chosenModel, candidateModels, chosenShortfall, stickyOverride - Per-class scores (scoreNeedsReasoning, scoreNoReasoning) - All fields from the existing unrestricted event
9cd119c to
672e30b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add a sendEnhancedGHTelemetryEvent call (automode.routerDecisionRestricted) alongside the existing unrestricted routerDecision event. The restricted event lands in copilot_v0_restricted_copilot_event in Hydro and includes:
hydraScores: JSON-encoded 4-dim capability scores (reasoning, code_gen, debugging, tool_use)
binaryScores: JSON-encoded binary classifier scores (needs_reasoning, no_reasoning)
chosenModel, candidateModels, chosenShortfall, stickyOverride
All fields from the existing unrestricted event (confidence, latency, routingMethod, etc.)
This enables joining router quality data with user prompts in a single Hydro table without cross-cluster joins to ddtelvscode. Only available for users opted into restricted telemetry (~44%).