feat: add chat summary generation model override - #26803
Conversation
Add a deployment-wide summary generation model override as a follow-up to persisted chat summaries. Co-authored-by: Cursor <[email protected]>
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Documentation CheckUpdates Needed
Automated review via Coder Agents |
… story The view now renders a 'Summary generation model' section between the title and explore sections. Update the AllOverridesUnset story so its exact level-3 heading assertion and per-section unset checks include the new section, fixing the failing Storybook interaction test.
Shorten the resolveChatSummaryModel and resolveSummaryGenerationModelOverride doc comments, condense the overrideSet test note to one line, and drop the redundant disabled-config note that the test name and argument already convey.
Resolve a conflict in resolveChatSummaryModel: the base now derives summary model options from the loaded transcript and dropped the runResult parameter. Resolve the chat model first to obtain provider keys, then apply the deployment summary override with those keys, mirroring the async title generation path.
johnstcn
left a comment
There was a problem hiding this comment.
I don't think we should add yet another configuration knob just for this. Can we just use the "small fast model" logic we already use for title generation, falling back to the chat model itself?
I would argue the chat summary override is more important that the title override that already exists. What if the same override was used for both title and summary? |
Actually, that opens up another can of worms. What if the user chooses a summary override that has too low a context window? |
They should get an error response, my goal was to allow users to have cost control of chat summaries. This could also be an issue if we choose the small fast model title generation uses. Right now chat summary defaults to the the model for the current chat. The question is if we really want to handle all edge cases for the user if they choose the wrong model. For example, we could fallback to a model with a larger context window. |
|
Another problem is that it looks like personal model were moved and this PR puts the override UI in the old location. |
…6657) Adds a persisted whole-chat summary that backs the chat summary popover. A new nullable `chats.summary` column is populated in the background after a successful root-chat turn and pushed to clients via a new `chat_summary_change` watch event (distinct from `summary_change`, which is bound to `last_turn_summary`), so the popover reads `chat.summary` straight off the loaded `Chat` with no extra query. This is the data source for the popover and per-chat cost UI built in #26649; the popover can consume `chat.summary` once this lands (the field is nullable, so merge order does not matter). ## How it works - **Generation** runs in the existing successful-turn finalize hook, detached from the request so the user's turn is never blocked. A cadence gate generates the first summary after one completed turn, then regenerates every three turns, using the `chats.summary_generated_at` freshness marker. Generation reads compaction-aware history, renders it to a bounded plain-text transcript (short transcripts are skipped), and asks for a 1-3 sentence summary via structured output. Failures never clear an existing summary. - **Staleness** is guarded by `history_version` (mirroring `last_turn_summary`), so a background write racing a newer turn loses while worker lifecycle transitions cannot reject a fresh write. - **Model selection** uses the chat's configured model. ## Deferred to follow-ups - **Cost accounting**: the `chat_messages.cost_source` discriminator and summary/title usage recording were removed from this PR so summary persistence is not blocked by hidden accounting rows advancing `history_version`. Title usage recording stays on main's `InsertChatMessages` path. - **Model override**: deployment-wide summary generation model selection is split into #26803; the base feature always uses the chat model. ## Notes - Migration `000540` adds `chats.summary` and `chats.summary_generated_at`, and recreates `chats_expanded` to expose the new columns. - Root chats only; shared viewers pick up the summary on their next refetch (live watch events are owner-only). Refs #26649 --------- Co-authored-by: Cursor <[email protected]>
Stacked on #26649. Adds the deployment-wide
summary_generationchat model override as a separate follow-up to persisted chat summaries and the summary popover.This mirrors the existing title-generation override path: admins can choose a summary generation model in Agent settings, the setting is stored in site config, and chat summary generation uses the configured model when available. If a configured override becomes unusable, summary generation is skipped until the override is changed or cleared.
Validation:
make gengo test ./coderd/x/chatd -run 'TestResolveSummaryGenerationModelOverride|TestResolveChatSummaryModel'go test ./coderd -run TestChatModelOverridespnpm checkfromsite/Made with Cursor