Codestin Search App

blockgroot · 2026-08-05T11:48:38Z

Disclosure: investigated and drafted with Claude Opus 5. I reviewed the change and ran the tests locally.

Streaming Responses interceptions recorded no token usage when the bridge was built with a nil mcp.ServerProxier, because recordTokenUsage was called from inside the i.mcpProxy != nil branch in aibridge/intercept/responses/streaming.go. Requests completed normally and returned 200, so traffic was served but metered as zero, with no error surfaced. Upstream reports usage on the response.completed event independently of tool injection, so the proxier is not a valid precondition for recording it.

That state is reachable in production: coderd/aibridged/pool.go:255-265 treats proxier construction failure as non-fatal ("MCP server injection can gracefully degrade") and caches the resulting bridge via SetWithTTL, so one transient config-retrieval error suppressed usage recording for every streaming Responses request served by that bridge until its TTL expired.

This records usage for every completed response, guarded only on completedResponse, matching responses/blocking.go and both chatcompletions implementations. Per-iteration semantics are preserved for the inner agentic loop.

The integration harness substituted a non-nil noop manager whenever no proxier was supplied (setupbridge.go:153-155), so the nil path was never exercised. withoutMCP() covers it.

Verification, with the fix reverted:

--- FAIL: TestResponsesStreamingRecordsTokenUsageWithoutMCP/without_mcp_proxy
        Error: "[]" should have 1 item(s), but has 0
--- PASS: TestResponsesStreamingRecordsTokenUsageWithoutMCP/with_noop_mcp_proxy

and with it applied:

--- PASS: TestResponsesStreamingRecordsTokenUsageWithoutMCP/without_mcp_proxy
--- PASS: TestResponsesStreamingRecordsTokenUsageWithoutMCP/with_noop_mcp_proxy
--- PASS: TestResponsesStreamingRecordsTokenUsagePerAgenticIteration

The per-iteration case asserts exactly two records for the injected-tool fixture, so decoupling the call from the proxier does not double-count when the agentic loop iterates. go test -race ./aibridge/... passes across all 15 packages.

Fixes #27885

One caveat on verification: I was unable to run make gen / make pre-commit locally, as I do not have the full mise toolchain installed. The change touches no codegen inputs (no SQL, protos, mocks, or TypeScript), so I do not expect generated-file drift, but flagging it rather than leaving it implied.

Streaming Responses interceptions called recordTokenUsage from inside the `i.mcpProxy != nil` branch, so a bridge built with a nil proxier served requests normally but recorded no token usage. Upstream reports usage on the response.completed event independently of tool injection. coderd/aibridged treats proxier construction failure as non-fatal and caches the resulting bridge, so a transient config-retrieval error suppressed usage recording for every streaming Responses request served by that bridge until its cache TTL expired. Record usage for every completed response, guarded only on completedResponse, matching responses/blocking.go and both chatcompletions implementations. Per-iteration semantics are preserved for the inner agentic loop. The integration harness substituted a non-nil noop manager whenever no proxier was supplied, so the nil path was never exercised. Add withoutMCP() to cover it. Fixes coder#27885

github-actions · 2026-08-05T11:48:49Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

blockgroot · 2026-08-05T12:10:28Z

I have read the CLA Document and I hereby sign the CLA

johnstcn · 2026-08-11T20:58:53Z

Hi @blockgroot, thank you for your contribution! Could you please double-check https://coder.com/docs/about/contributing/AI_CONTRIBUTING

blockgroot · 2026-08-12T06:26:47Z

Thanks @johnstcn, and apologies for not including this up front. I've added a disclosure to the PR description per AI_CONTRIBUTING: this was investigated and drafted with Claude Opus 5, and I reviewed the change and ran the tests locally.

Summarising the verification that's in the description:

TestResponsesStreamingRecordsTokenUsageWithoutMCP/without_mcp_proxy fails without the fix ("[]" should have 1 item(s), but has 0) and passes with it, while the with_noop_mcp_proxy control passes either way. The only difference between the two cases is whether the proxier is nil.
TestResponsesStreamingRecordsTokenUsagePerAgenticIteration asserts exactly two records for the injected-tool fixture, so moving the call out of the MCP branch does not double-count when the inner loop iterates.
go test -race ./aibridge/... passes across all 15 packages, and golangci-lint (v1.64.8, the version pinned in mise.toml), paralleltestctx, intxcheck and make lint/emdash are clean.

One gap I should flag: I couldn't run make gen / make pre-commit, as I don't have the full mise toolchain installed locally. The change touches no codegen inputs, so I don't expect drift, but I didn't want to imply I'd run checks that I hadn't.

Happy to make any changes that would help review, or to drop the harness option if you'd prefer the nil case covered a different way.

johnstcn

Thanks for the update @blockgroot! I have some comments below.

johnstcn · 2026-08-12T07:40:03Z

+// withoutMCP runs the bridge with a nil MCP server proxier, matching a
+// deployment where proxier construction failed and injection degraded. This
+// differs from NoopMCPManager, which is non-nil and reports zero tools.
+func withoutMCP() bridgeOption {
+	return func(c *bridgeConfig) { c.noMCPProxy = true }
+}
+


Suggestion: since this is just used by one test, we could just inline it where it's used.

Done in 0e81277. The option is now inlined at its only call site as func(c *bridgeConfig) { c.noMCPProxy = true }.

I kept the noMCPProxy field itself, since newBridgeTestServer still needs to distinguish "no proxier specified" from "explicitly nil" at setupbridge.go:156, and moved the explanation onto the field.

johnstcn · 2026-08-12T07:59:02Z

+			require.Positive(t, bridgeServer.Recorder.TotalInputTokens())
+			require.Positive(t, bridgeServer.Recorder.TotalOutputTokens())


We should be comparing this with the token counts before the request.

Done in 0e81277. The test now captures TotalInputTokens() and TotalOutputTokens() before the request and asserts the delta.

I also switched from require.Positive to the exact fixture totals, matching the style in TestOpenAIChatCompletions (bridge_internal_test.go:820), since that catches miscounting rather than just absence.

johnstcn · 2026-08-12T08:01:30Z

+// TestResponsesStreamingRecordsTokenUsagePerAgenticIteration asserts that
+// decoupling token recording from the MCP proxier does not double-count usage
+// when the inner agentic loop iterates. The injected-tool fixture drives two
+// upstream calls, so two records are expected.
+func TestResponsesStreamingRecordsTokenUsagePerAgenticIteration(t *testing.T) {
+	t.Parallel()
+
+	bridgeServer, _, resp := setupInjectedToolTest(
+		t,
+		fixtures.OaiResponsesStreamingSingleInjectedTool,
+		true,
+		defaultTracer,
+		pathOpenAIResponses,
+		nil,
+	)
+	defer resp.Body.Close()
+
+	usages := bridgeServer.Recorder.RecordedTokenUsages()
+	require.Len(t, usages, 2, "one token usage record per agentic iteration")
+}


I'd suggest moving this test into TestOpenAIChatCompletions. Also, do we need an equivalent test for the Anthropic provider?

Both addressed in 0e81277, though not quite as suggested. Two things:

On the agentic-iteration test: I removed it entirely rather than moving it. TestResponsesInjectedTool/streaming_success (responses_internal_test.go:872) already drives the same OaiResponsesStreamingSingleInjectedTool fixture and asserts an exact record count via require.Len(t, tokenUsages, len(tc.expectTokenUsages)) at :987, so the double-counting case was already covered. Mine was redundant.

On placement: I moved the remaining test into bridge_internal_test.go and deleted the standalone file. I didn't put it inside TestOpenAIChatCompletions because it exercises /openai/v1/responses and /anthropic/v1/messages rather than /openai/v1/chat/completions. Happy to move it if you'd still prefer it grouped there.

On Anthropic: added as a second case in the same table. Worth noting it passes with and without the fix, so it's a parity guard rather than a regression test. The Messages implementation records usage inside the SSE event switch at messages/streaming.go:269 (message_start) and :294 (message_delta), neither gated on the proxier. The mcpProxy checks in that function only decide whether events are relayed downstream, and they run after recording. So Anthropic never had this bug, but the case now pins that behaviour.

Inline the nil-proxier bridge option at its only call site and document the noMCPProxy field, which newBridgeTestServer still needs to tell "unset" apart from "explicitly nil". Compare token counts against the values recorded before the request, and assert the exact fixture totals rather than only that they are positive, matching TestOpenAIChatCompletions. Fold the standalone test file into bridge_internal_test.go and cover the Anthropic Messages route alongside OpenAI Responses. Drop the per-agentic-iteration test: TestResponsesInjectedTool/streaming_success already drives the same fixture and asserts an exact record count, so it covers the double-counting case.

johnstcn · 2026-08-12T12:45:20Z

@blockgroot Thanks for addressing the feedback! You may need to merge latest main to address the Storybook CI job.

blockgroot · 2026-08-12T14:19:46Z

Updated, now at 07d29f2.

Storybook still fails with the same PIXEL_KEY error. Its job spec, pixel.jsonc, and the pixel-storybook version are unchanged on main, and fork PRs don't get repo secrets, so this isn't a staleness issue. May need a fork exception in that job.

Also, ci is gated on action_required, likely pending maintainer approval.

github-actions Bot added the community Pull Requests and issues created by the community. label Aug 5, 2026

github-actions Bot assigned blockgroot Aug 5, 2026

cdrci2 added a commit to coder/cla that referenced this pull request Aug 5, 2026

@blockgroot has signed the CLA in coder/coder#27886

1819552

blockgroot marked this pull request as ready for review August 5, 2026 12:11

johnstcn self-requested a review August 12, 2026 07:40

johnstcn reviewed Aug 12, 2026

View reviewed changes

Merge branch 'main' into fix/aibridge-responses-token-usage-without-mcp

07d29f2

johnstcn approved these changes Aug 12, 2026

View reviewed changes

johnstcn merged commit 7a545b3 into coder:main Aug 12, 2026
27 of 28 checks passed

github-actions Bot locked and limited conversation to collaborators Aug 12, 2026

		require.Positive(t, bridgeServer.Recorder.TotalInputTokens())
		require.Positive(t, bridgeServer.Recorder.TotalOutputTokens())

Uh oh!

Conversation

blockgroot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blockgroot commented Aug 5, 2026

Uh oh!

johnstcn commented Aug 11, 2026

Uh oh!

blockgroot commented Aug 12, 2026

Uh oh!

johnstcn left a comment

Choose a reason for hiding this comment

Uh oh!

johnstcn Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

blockgroot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

johnstcn Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

blockgroot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

johnstcn Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

blockgroot Aug 12, 2026

Choose a reason for hiding this comment

Uh oh!

johnstcn commented Aug 12, 2026

Uh oh!

blockgroot commented Aug 12, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

blockgroot commented Aug 5, 2026 •

edited

Loading

github-actions Bot commented Aug 5, 2026 •

edited

Loading