Add requestKind to chat response telemetry events#312262
Open
kevin-m-kent wants to merge 1 commit intomicrosoft:mainfrom
Open
Add requestKind to chat response telemetry events#312262kevin-m-kent wants to merge 1 commit intomicrosoft:mainfrom
kevin-m-kent wants to merge 1 commit intomicrosoft:mainfrom
Conversation
Emit a new 'requestKind' property on response.success, response.cancelled, and response.error. Values: 'background', 'subagent', or 'normal' (derived from the same requestKindOptions that sets the X-Interaction-Type header). This lets us distinguish background and subagent requests from primary user requests in telemetry, complementing the existing initiatorType (user/agent) signal. Co-authored-by: Copilot <[email protected]>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR extends Copilot chat response telemetry to include a requestKind dimension ('normal' | 'background' | 'subagent') on the CAPI path, derived from existing requestKindOptions that already influence the X-Interaction-Type header. This enables slicing success/cancel/error metrics by request kind without changing runtime behavior.
Changes:
- Adds
requestKindOptionsto the telemetry payload interfaces and emitsrequestKindonresponse.success,response.cancelled, andresponse.error. - Updates the GDPR schemas for the three events to include
requestKind(SystemMetaData / FeatureInsight). - Threads
requestKindOptionsthroughChatMLFetcherImplsuccess/cancel/error telemetry call paths (including retry/error paths).
Show a summary per file
| File | Description |
|---|---|
| extensions/copilot/src/extension/prompt/node/chatMLFetcherTelemetry.ts | Adds requestKind emission and GDPR schema entries for success/cancel/error response telemetry. |
| extensions/copilot/src/extension/prompt/node/chatMLFetcher.ts | Passes requestKindOptions through to telemetry senders and into processSuccessfulResponse. |
Copilot's findings
- Files reviewed: 2/2 changed files
- Comments generated: 0
Contributor
Author
|
@roblourens curious what you think of this. We are looking to filter out background requests in a robust way for our A/Bs, rather than maintain a list of sources. Copilot seems to think that this header for background interaction type never actually fires, but I'm suspicious of that claim. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Emit a new
requestKindproperty onresponse.success,response.cancelled, andresponse.errortelemetry events. Values are'background','subagent', or'normal', derived from the samerequestKindOptionsthat already sets the outgoingX-Interaction-Typeheader in CAPI requests.Motivation
Today we can tell from telemetry whether a chat request was initiated by the user vs. an agent tool call (via
initiatorType), but we can't distinguish which agent requests are background (speculative/system) vs. subagent (spawned by a parent agent) vs. normal primary-turn requests. That signal is already on the wire as theX-Interaction-Typeheader, but it isn't logged client-side. This makes it hard to slice perf / success metrics by request kind.Changes
extensions/copilot/src/extension/prompt/node/chatMLFetcherTelemetry.tsrequestKindOptionstoIChatMLFetcherSuccessfulData,IChatMLFetcherCancellationProperties,IChatMLFetcherErrorData.requestKind: requestKindOptions?.kind ?? 'normal'onresponse.success,response.cancelled,response.error.__GDPR__schema (classificationSystemMetaData, purposeFeatureInsight).extensions/copilot/src/extension/prompt/node/chatMLFetcher.tsrequestKindOptions(already destructured at the top offetchMany) into all five call sites: the success, cancel, and error calls infetchManyand_retryAfterError, plus the newprocessSuccessfulResponseparameter.Notes
initiatorTypein theirresponse.successGDPR schema but don't actually emit it today; intentionally leaving BYOK out of this PR to keep it scoped to the CAPI path.