fix(coderd): exclude accounting rows from chat history_version - #26689
fix(coderd): exclude accounting rows from chat history_version#26689jaaydenh wants to merge 11 commits into
Conversation
Add a persisted chats.summary populated by a background generator after successful root-chat turns, delivered live via a new chat_summary_change watch event, with per-feature cost attribution and a deployment-wide summary-generation model override.
The UnknownContextReturns400 subtest hardcodes the valid override context list in its expected error. Adding the summary_generation context changed the message, so update both assertions.
|
/coder-agents-review |
|
Chat: Review posted | View chat Review historydeep-review v0.9.0 | Round 3 | Last posted: Round 3, 10 findings (1 P0, 2 P3, 3 P4, 3 Nit, 1 Note), APPROVE. Review Finding inventoryFindings
Contested and acknowledgedCRF-7 (P4, 000535_up.sql:32) - cost_source discriminator has no schema-level constraint
CRF-8 (Nit, commit) - Commit scope too narrow
Round logRound 1Panel. Netero + 15 reviewers (bisky, chopper, ging-go, gon, hisoka, kite, knuckle, komugi, leorio, mafu-san, mafuuu, meruem, pariston, knov, zoro). 1 P3, 2 Nit, 3 P4, 1 Note. Reviewed against 0bc176d..9ad21cb. Round 2Churn guard: PROCEED. 6 addressed, 2 acknowledged, 0 silent. Panel (bisky, hisoka, mafu-san, mafuuu, pariston, kite, knuckle, komugi, knov). 1 P0, 1 P3 new. Reviewed against 877bbf0..2cf46f3. Round 3Churn guard: PROCEED. 2 addressed, 0 silent. Panel (bisky, hisoka, mafu-san, mafuuu, pariston, kite, knov). 0 new findings. All 10 findings resolved or acknowledged. CI green. Reviewed against 877bbf0..785ef29. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9ad21cb1e2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Clean, well-targeted fix. The root cause analysis is correct: accounting rows were advancing history_version because (1) the AFTER STATEMENT triggers had no filter for non-history rows and (2) the two-step insert made the trigger blind to cost_source. The fix addresses both: trigger filter on cost_source IS NULL and atomic insert via InsertChatAccountingMessage. The removal of the last_model_config_id restoration is a structural simplification, not cosmetic.
Severity breakdown: 1 P3, 3 P4, 2 Nit, 1 Note.
Knuckle confirmed the migration is safe for online deployment: CREATE OR REPLACE FUNCTION rewrites pg_proc without acquiring ACCESS EXCLUSIVE on chat_messages or chats. No table rewrite, no lock queue.
Pariston traced the full causal chain and evaluated four alternative framings (trigger filter gap, two-step atomicity, guard scope mismatch, separate accounting table). The chosen approach (framings 1+2) is proportional. Pariston: "I tried to build a case against this and couldn't."
The trigger test (TestAccountingMessageDoesNotAdvanceHistoryVersion) is well-structured with three independent phases and a positive control. Both mock tests pass. DB integration tests cannot run due to the duplicate migration 000530 from the stacked base branch (#26657); this resolves when that PR renumbers before merge.
Leorio, on the migration header comment: "A developer touching this trigger a year from now gets the full diagnosis without the investigation. Every migration and query comment in this codebase should read like these."
🤖 This review was automatically generated with Coder Agents.
# Conflicts: # docs/admin/security/audit-logs.md
Resolve the actionable findings from the deep review of #26657: - P0/CRF-2: renumber the chat summary migration from 000530 to 000534 to avoid colliding with 000530_relay_host_nats_port on main. - CRF-3: scope summary_change/chat_summary_change merges to their own field so an equal-timestamp event cannot clobber the other summary; add the equal-timestamp regression tests. - CRF-4/CRF-13: bail out of background summary generation when shutdown has begun so Close() is not blocked, and fix the stale comment. - CRF-5: re-read the chat before the cadence gate so rapid turns do not both pass against a stale snapshot. - CRF-7: add a CHECK constraint on chat_messages.cost_source. - CRF-8: use a dedicated chatSummaryWriteTimeout for the summary write. - CRF-17: only count sentence terminators at a word boundary so dotted identifiers do not inflate the count. - CRF-1/CRF-6: deduplicate the usage-recording and model-override resolution paths shared by title and summary generation. - CRF-15/CRF-16: cover ChatMessageVisibilityUser and model-only user messages in the summary tests. - CRF-9/10/11/12/19/22: comment cleanups, errors.AsType, and drop the popover reference from the settings description.
CRF-23: add TestCountSentenceTerminators so the CRF-17 fix is covered. Without it, removing the word-boundary guard in countSentenceTerminators would not fail any test. The test asserts periods inside dotted identifiers are not counted and that a dotted-identifier-dense summary stays under summaryMaxSentences.
Merge conflict resolved + review feedback addressedConflict resolution (merge Review findings (fixes in
|
Move summary generation model overrides out of the persisted summary PR so the base feature always uses the chat model. Co-authored-by: Cursor <[email protected]>
Remove cost_source, UpdateChatMessageCostSource, and summary usage recording from the whole-chat summary feature so summary persistence is not blocked by hidden accounting rows advancing history_version. Title usage recording reverts to main's InsertChatMessages path. Co-authored-by: Cursor <[email protected]>
Stack on the simplified chat-summary branch that defers cost accounting. This PR adds cost_source, InsertChatAccountingMessage, and migration 000535 recreating history triggers so only ordinary turn rows advance history_version. Summary and title usage writers record spend via the atomic accounting insert so background summary persistence is not invalidated by its own usage row. Co-authored-by: Cursor <[email protected]>
b13db9f to
2cf46f3
Compare
|
/coder-agents-review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2cf46f3266
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
All R1 findings addressed or acknowledged. Good work on the shared recordHiddenUsageMessageTx helper (resolves CRF-3, CRF-4, CRF-6), the CHECK constraint on cost_source (resolves CRF-1), and the new summary path test (resolves CRF-5).
This round found one P0 and one P3.
The base branch merge overwrote the InsertChatAccountingMessage dbauthz implementation with a panic("not implemented") stub (CRF-9). The dbauthz test is also missing. This affects every production call path. The mock tests pass because they bypass dbauthz.
Note: the PR description's "verified against a dev instance" claim likely did not exercise the cost attribution path. If the dev model returned zero usage, the if usage != (fantasy.Usage{}) guard at chatd.go:5022 skips recordChatSummaryUsage entirely, so the panic was never hit. After fixing the dbauthz implementation, consider verifying that usage recording is exercised end-to-end.
Pariston: "I tried to build a case against this change and could not. The problem is correctly diagnosed. The two root causes (trigger not filtering, cost_source set after INSERT) are both real and both addressed."
🤖 This review was automatically generated with Coder Agents.
…nscript The detached background summary goroutine re-reads the chat and transcript fresh, but resolveChatSummaryModel still used the ModelBuildOptions captured from the turn that launched the goroutine. In AI Gateway deployments that ActiveAPIKeyID can be stale or belong to a different turn once a later turn commits first, so the summary call could fail with a missing key or be attributed to the wrong key. Derive modelBuildOptionsFromMessages(messages) after loading the transcript and thread the active turn API key onto the context, mirroring deriveFinalTurnRunResult and the generation path. With this, runResult is no longer needed by the summary helpers, so drop the parameter. Also fix the generateAndStoreChatSummary doc comment (cost recording is deferred to #26689) and add a shouldGenerateChatSummary subtest that pins the countCompletedTurnsSince time-filter guard against pre-marker turns.
The base-branch merge overwrote the InsertChatAccountingMessage dbauthz
method with a panic("not implemented") stub. Production paths are
dbauthz-wrapped, so recordChatSummaryUsage and recordManualTitleUsage
would panic at runtime. Restore the authorization (authorize update on
the parent chat, delegating to the underlying store) and the dbauthz
test.
Also correct the snapshot_version note in ARCHITECTURE.md and the 000535
migration comment: accounting inserts do not bump snapshot_version, so an
accounting-only write leaves both version columns unchanged.
|
/coder-agents-review |
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 785ef29099
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| } | ||
|
|
||
| model, _, ok := p.resolveChatSummaryModel(authCtx, chat, runResult, logger) | ||
| model, modelConfig, ok := p.resolveChatSummaryModel(authCtx, chat, runResult, logger) |
There was a problem hiding this comment.
Derive summary key from the transcript before calling
In AI Gateway chats with rapid back-to-back turns, this resolves the summary model with runResult.ModelBuildOptions from the turn that scheduled the goroutine, while the new accounting code records usage against the active key found in the freshly loaded messages. If a newer turn committed before this goroutine ran, the summary request is sent with the old delegated key but the usage row is attributed to the new key, which can fail routing or misattribute spend. Derive the model options from the same loaded transcript and use that key for both the model and recordChatSummaryUsage.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
All 10 findings from rounds 1 and 2 are resolved. CI is green. No new findings from Netero or the R3 panel (7 reviewers).
CRF-9 (P0, dbauthz panic) verified fixed: InsertChatAccountingMessage now authorizes ActionUpdate on the parent chat, matching the established pattern. CRF-10 (P3, ARCHITECTURE.md inaccuracy) verified fixed: doc no longer claims snapshot_version advances on accounting inserts.
Mafu-san: "Fixed siblings without being asked. CRF-10 flagged a false snapshot_version claim in ARCHITECTURE.md. The R3 fix corrected it in both ARCHITECTURE.md and the migration comment, which contained the same false claim but was not called out separately."
This is a clean, well-targeted fix. The root cause is correctly diagnosed, the solution is proportional, and the test coverage spans all three layers (trigger, query, application). The recordHiddenUsageMessageTx helper is a structural improvement over the old duplicated 4-step pattern.
🤖 This review was automatically generated with Coder Agents.
Trim the verbose, duplicated rationale comments added for the chat accounting work down to short, single-purpose comments. The full explanation of why accounting rows must not advance history_version now lives once in coderd/x/chatd/ARCHITECTURE.md; the code, SQL, migration, and test comments keep only the non-obvious essentials.
Stacked on #26657 (the persisted whole-chat summary backend). Base branch is
chat-summary-62j9; review/merge that first.Fixes a bug where the persisted whole-chat summary (and the existing
last_turn_summarysidebar label) was generated after a turn but never saved, so the summary popover always showed "No summary yet."The summary write is guarded by an optimistic check on
chats.history_version(UPDATE chats SET summary=... WHERE history_version = @expected). Background summary and manual title generation record their spend as hidden, soft-deletedchat_messagesaccounting rows. Two issues made those accounting rows advancehistory_versionand invalidate the guard:history_versionfor every message row, including accounting rows.InsertChatMessageswithcost_sourceNULL and only tagged afterward byUpdateChatMessageCostSource, so the insert trigger saw an ordinary row regardless.This PR:
000531recreating the history triggers so only rows withcost_source IS NULLadvancehistory_version.InsertChatAccountingMessage, which setscost_sourceon the initial INSERT so the trigger classifies the row correctly, and switches the summary and title usage writers to it.UpdateChatMessageCostSourcequery and thelast_model_config_idrestoration in those accounting paths.snapshot_versionstill advances for every change, sohistory_versiontrails after an accounting-only write and catches up on the next real message.Verified against a dev instance: a fresh chat now persists its summary after the first completed turn, with no
skipped stale chat summary update.