Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: back the per-chat cost endpoint with AI Gateway data - #27328

Merged
ibetitsmike merged 10 commits into
mainfrom
mike/chat-cost/gateway-cost-api
Jul 30, 2026
Merged

feat: back the per-chat cost endpoint with AI Gateway data#27328
ibetitsmike merged 10 commits into
mainfrom
mike/chat-cost/gateway-cost-api

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Stack Context

This stack removes native chat cost tracking and native chat usage limits, making the AI Gateway the single source of AI spend data and budget enforcement.

  1. This PR: re-back the per-chat cost endpoint with AI Gateway data.
  2. Remove native chat usage limits end to end, rewiring the sidebar indicator to gateway spend.
  3. Remove native chat cost tracking end to end, deleting the Analytics/Spend cost UI.

What?

GET /api/experimental/chats/{chat}/cost summed chat_messages.total_cost_micros, which native chat cost tracking maintained. It now aggregates AI Gateway interception data instead, and has no native fallback.

  • New GetAIBridgeChatCost query, authorized through the root chat so members can read their own chat's cost without gaining access to raw interception rows.
  • Response fields renamed: priced_message_count -> request_count, unpriced_messages_having_usage_count -> unpriced_request_count.
  • The chat summary sidebar keys its cost cache by root chat, and hides the cost row where the AI Gateway is off or unlicensed. The root cost is invalidated when a chat leaves an active status and when a generated title lands, since title generation bills its own gateway request.

GetChatModelUsageCostByChatID and the rest of native cost tracking are untouched here; PR 3 removes them.

Why?

Native cost tracking duplicates what the AI Gateway already records, and the two disagree. Repointing the endpoint first means the cost UI keeps working while the native implementation is deleted later in the stack.

Two behaviour changes follow from gateway semantics and are intentional:

  • Requests, not messages. The gateway records interceptions, so counts are requests. Title-generation traffic now counts.
  • Whole-tree totals. The gateway records the spawning chat's ID as the interception session ID, so a subagent's requests are attributed to its immediate parent, not always the root. Only a whole chat tree can be summed, so the query resolves the root and aggregates the tree, and every chat in a tree reports the same total. Native returned per-subtree totals.

Attribution and counting semantics

The aggregate groups token usage per interception before counting, so the reported numbers are per request even though a request records one usage row per provider response:

  • RequestCount counts finished Coder Agents interceptions in the tree, including unpriced ones.
  • UnpricedRequestCount counts requests with at least one usage row the gateway could not price. It is a subset of RequestCount.
  • TotalCostMicros omits only unpriced usage, so a partially priced request still contributes its priced portion. The sidebar therefore says Excludes unpriced usage from N request(s) rather than claiming whole requests were dropped.

A recorded cost of zero is a free request, not an unpriced one. Usage without an effective group is excluded, matching what never reached ai_user_daily_spend.

Authorization

Reads go through ExtractChatParam plus ResourceChat, with no cost-specific RBAC widening. TestGetChatCost/MemberCanReadOwnChat covers a scoped agents-access member reading their own chat's cost, and MemberCannotReadOtherUsersChat still asserts 404 for a non-owner. Plain members without agents-access cannot create or read chats at all, so they never reach this endpoint.

Known limitation

AI Gateway data has its own retention period, 60 days by default and configured independently of chat retention, so spend for requests older than that is no longer reported. A chat whose gateway records have all been purged reports zero cost, which is indistinguishable from genuinely free usage under this contract. The endpoint documents the caveat; #27330 documents it on the Spend Management page.

In-flight interceptions are excluded, since cost is only known once the response is recorded. A chat's cost therefore lags the active turn by one request.

Rebase note

Rebased onto main after #27579 removed the ai-gateway-cost-control experiment. The per-chat cost row is now gated on the aibridge feature alone, matching how #27579 degated the other cost-control surfaces.

Mux prepared this PR on Mike's behalf.

@github-actions

github-actions Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 955196e618

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/pages/AgentsPage/AgentAnalyticsPage.tsx Outdated
Comment thread site/src/pages/AgentsPage/AgentAnalyticsPage.tsx Outdated
Comment thread enterprise/coderd/aibridge.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5aae0e1845

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread enterprise/coderd/aibridge.go Outdated
Comment thread site/src/pages/AISettingsPage/SpendPage/components/SpendDrillInView.tsx Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 65369cd937

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/dbauthz/dbauthz.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 961d7d0a57

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/pages/AISettingsPage/SpendPage/SpendPage.tsx Outdated
Comment thread coderd/database/queries/aibridge.sql Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fdcfe50486

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread coderd/database/queries/aibridge.sql Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f427c6f957

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread coderd/database/dbauthz/dbauthz.go Outdated
Comment thread enterprise/coderd/aibridge.go Outdated
Comment thread site/src/pages/AgentsPage/AgentAnalyticsPageView.tsx Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c0b3da4b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/aibridge.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7c0b3da4b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread enterprise/coderd/aibridge.go Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

1 similar comment
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 🎉

Reviewed commit: 0afa8404cf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike force-pushed the mike/chat-cost/gateway-cost-api branch from 0afa840 to 5468559 Compare July 20, 2026 09:25
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: 5468559aa8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike force-pushed the mike/chat-cost/gateway-cost-api branch from 5468559 to 0389c65 Compare July 22, 2026 23:38

Copy link
Copy Markdown
Collaborator Author

Comment thread coderd/database/queries/aibridge.sql Outdated
@ibetitsmike
ibetitsmike force-pushed the mike/chat-cost/gateway-cost-api branch from 0389c65 to 3885435 Compare July 27, 2026 12:18
GetAIBridgeChatCost now aggregates token usage per interception in a CTE
instead of relying on COUNT(DISTINCT) over the joined rows. The reported
counts are unchanged, but a request that records several provider
responses is now visibly aggregated once, and the unpriced flag reads as
'this request had usage we could not price'.

Correct the SDK doc and the sidebar copy to match: a partially priced
request still contributes its priced usage to the total, so the note now
says the total excludes unpriced usage rather than whole requests.

Add coverage for a partially priced request, a request priced at zero, a
sibling root chat tree, and a member reading the cost of a chat they own.
…rol experiment gate

The ai-gateway-cost-control experiment was removed upstream, so gate the
per-chat cost row on the aibridge feature alone.
@ibetitsmike
ibetitsmike force-pushed the mike/chat-cost/gateway-cost-api branch from cc9086e to e7a7fba Compare July 29, 2026 17:30
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

Force-pushed after rebasing the stack onto main. New head: e7a7fbad857.

Upstream #27579 removed the ai-gateway-cost-control experiment, which this stack still gated on. Git merged cleanly but the result did not compile, so CI on #27328 failed lint, check-build, test-e2e, and Storybook with Type '"ai-gateway-cost-control"' is not assignable to type 'Experiment'. The descendant PRs passed only because their bases are stack branches rather than main.

Change here: the per-chat cost row is gated on the aibridge feature alone, matching how #27579 degated the other cost-control surfaces, plus one stale story comment corrected.

Validated per branch: go build ./..., go vet ./... (no new findings), pnpm run lint:types, make lint, make gen clean, TestGetChatCost, and the affected Storybook and unit tests. Earlier review approvals were given on the pre-rebase heads.

Mux is acting on Mike's behalf.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Mux is acting on Mike's behalf.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7a7fbad85

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/reference/api/chats.md
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e7a7fbad85

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/database/queries/aibridge.sql Outdated

@johnstcn johnstcn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backend looks good to me, just one test coverage gap I see. Deferring explicit approval to Susana and Danielle.

Comment thread coderd/exp_chats_internal_test.go Outdated
A request that fails upstream still ends but records no token usage, so the
inner join dropped it from request_count. Left-join eligible usage instead,
and guard the unpriced check on the usage row existing so a request with no
usage is not reported as unpriced.

@DanielleMaywood DanielleMaywood left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving Approval to Susana but otherwise the frontend is fine

Comment thread site/src/pages/AgentsPage/components/ChatSummaryPanel.stories.tsx Outdated
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e225bccacf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats.go
The cost endpoint only totals retained AI Gateway records, so state the
retention caveat where the endpoint is introduced rather than later in the
stack. Also assert UnpricedRequestCount in the root-resolve test and inline
the single-use story parameters constant.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6a706b5753

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread coderd/exp_chats.go

@ssncferreira ssncferreira left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

chats.parent_chat_id and chats.root_chat_id are both ON DELETE SET NULL, so
deleting a root leaves descendants with only a parent. Resolve the cost tree
with the same COALESCE(root_chat_id, parent_chat_id) precedence the chat
queries use, on both the endpoint and the frontend cache key.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 432b4b16b5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread site/src/pages/AgentsPage/AgentsPageLayout.tsx Outdated
…t tree

The cost cache key and the watch-event invalidation key resolved the tree
differently, so an event on a chat whose root was deleted left a mounted cost
stale. Both now use one getChatCostTreeID helper, which also trims blank IDs
that a nullish check would accept.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 5ce000c0bb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike merged commit 95a2c2b into main Jul 30, 2026
33 checks passed
@ibetitsmike
ibetitsmike deleted the mike/chat-cost/gateway-cost-api branch July 30, 2026 11:01
@github-actions github-actions Bot locked and limited conversation to collaborators Jul 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants