feat(coderd/x/chatd): bill interrupted tool batches on cancellation rows - #28362
Merged
jaaydenh merged 4 commits intoAug 25, 2026
Merged
Conversation
This was referenced Aug 20, 2026
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 20, 2026 09:11
4ec0b24 to
709c3be
Compare
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 20, 2026 11:28
1f0f883 to
dea7b77
Compare
Contributor
Author
|
@codex review |
jaaydenh
marked this pull request as ready for review
August 20, 2026 11:50
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dea7b77830
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 20, 2026 13:05
dea7b77 to
a9a92e7
Compare
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 21, 2026 09:25
a9a92e7 to
e9762eb
Compare
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 22, 2026 07:06
e9762eb to
c7f822d
Compare
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 24, 2026 09:52
a306d1a to
fb38522
Compare
4 tasks
hugodutka
approved these changes
Aug 24, 2026
spikecurtis
approved these changes
Aug 25, 2026
jaaydenh
added a commit
that referenced
this pull request
Aug 25, 2026
…ocalTools (#28359) **Stack**: **#28359 (this PR)** → #28360 → #28361 → #28362 `chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so callers had to reach through `outcome.Step` for every field. `ExecuteLocalTools` now returns the `PersistedStep` directly and the wrapper type is deleted. No behavior change. First of a four-PR stack that makes local tool execution count toward Coder Agent runtime (splitting #28211). This refactor keeps the rename noise out of the feature PRs. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).
jaaydenh
added a commit
that referenced
this pull request
Aug 25, 2026
) **Stack**: #28359 → **#28360 (this PR)** → #28361 → #28362 ## Summary Completed local tool batches now count toward Coder Agent runtime (`chat_messages.runtime_ms`, the source of truth for `hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid double counting, and without multiplying runtime for parallel tool calls. Second of a four-PR stack splitting #28211. Stacked on #28359. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime). ## Problem Agent runtime previously measured only model invocation wall clock (stream open to fully consumed). Time spent executing local tools between steps, including file operations, terminal commands, workspace provisioning, and MCP tools, was deliberately excluded, which undercounts the product definition of "actively processing a task". Naive inclusion has two hazards: a batch of parallel tool calls would bill N windows for one wall-clock wait, and `wait_agent` would re-bill child agents that already bill their own model and tool time. ## Fix Each completed local tool batch bills one window: the union of the billed tools' execution intervals, persisted as `runtime_ms` on a dedicated usage record appended after the batch's tool-result rows. The record is a tool-role message with `visibility='model'`, so it never reaches the API, SSE, or clients, and prompt replay drops it because its single internal `tool-batch-usage` part converts to no provider content. Its content carries an audit payload (`billed_ms`, `billed_calls`) so the billed window is inspectable after the fact; real tool-result rows never carry batch-level runtime. Concurrent calls all start at batch start, so 5 parallel 10s reads bill 10s, not 50s; serial calls (`SerialToolCalls`) count only from their own launch, so unbilled waits before them do not count. Sub-agent orchestration tools (`spawn_agent`, `wait_agent`, `message_agent`, `interrupt_agent`, `list_agents`, `list_subagent_models`, plus the deprecated `close_agent` alias) never extend the window: every chat, including children, applies the same rules to its own runtime, so a parent's `wait_agent` window would double count. A lone `wait_agent` bills 0 and appends no usage record; `execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins the unbilled set to the registered sub-agent tool catalog so they cannot drift. `GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms` role-agnostically, so the usage record is picked up with no schema, query, or cron changes. Client-executed dynamic tools, external agents, parked/idle time, and retry backoff remain unbilled. Interrupted batches still bill only the model window; the rest of the stack adds partial-window billing via the same usage record. `ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder` (`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order execution stamps. This PR wires no recorder; the next PR in the stack connects it to the message part buffer for interrupt billing. **NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from deploy forward. There is no backfill and no feature flag.
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
2 times, most recently
from
August 25, 2026 09:56
ea3720b to
f6adb9f
Compare
Base automatically changed from
jaayden/codagt-928-stack-3-buffer-tool-execution-stamps
to
main
August 25, 2026 09:57
jaaydenh
added a commit
that referenced
this pull request
Aug 25, 2026
…e part buffer (#28361) **Stack**: #28359 → #28360 → **#28361 (this PR)** → #28362 During local tool execution, buffer episodes now record when each tool-call occurrence actually starts and finishes, via `RecordToolStart`/`RecordToolCompletion`/`ToolCompletions` on the message part buffer. Occurrences are keyed by unresolved-call position, so gaps left by rejected calls and duplicate tool-call IDs stay distinct: `partitionAmbiguousToolCalls` additionally returns the allowed calls' input positions, and generation's `chatloop.ToolBillingRecorder` implementation translates dispatch-order stamps back through them. Mechanism only: nothing reads the stamps yet. The final PR in the stack uses them to bill interrupted tool batches on a dedicated usage record committed alongside their synthesized cancellation rows. Third of a four-PR stack splitting #28211. Stacked on #28360. Refs CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime). --------- Co-authored-by: Hugo Dutka <[email protected]>
Interrupting a local tool batch previously billed only the model window; work the tools already performed was lost. The interrupt task now snapshots the episode's billing state (model invocation, live tool stamps, buffered parts, and the interrupt instant) before its first database read, and reuses that snapshot across task retries so buffer eviction cannot lose it and retries cannot inflate it. Synthesized cancellation rows bill the union of the started, billable calls' partial execution intervals, placed once on the row whose interval ends last. Unstarted, rejected, and sub-agent orchestration calls bill nothing. The interrupted model window now also ends at the snapshot instant instead of after the chat read.
… the buffer The retry-stable interrupt snapshot guarded one rare case: an interrupt task retrying after its episode was closed and garbage collected (a database stall longer than the buffer's 15-second retention) would bill nothing. That undercount matches the existing crash-recovery precedent and is not worth the machinery, per review on the original PR. StartInterrupt now reads ModelInvokedAt and ToolCompletions inline just before closing the episode, and reuses the single interrupt instant for cancellation-row billing so transaction delay cannot inflate windows.
…d usage record Match the completed-batch billing shape: instead of stamping the interval union on the cancellation row whose interval ends last, append the shared batchUsageMessage record carrying the billed window and an audit payload. Cancellation rows stay free of batch-level runtime.
jaaydenh
force-pushed
the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
from
August 25, 2026 09:57
f6adb9f to
32e9e8e
Compare
Contributor
Documentation CheckUpdates Needed
Automated review via Coder Agents |
jaaydenh
deleted the
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
branch
August 25, 2026 10:29
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack: #28359 → #28360 → #28361 → #28362 (this PR)
Summary
Interrupted local tool batches now bill the partial window they actually executed, recorded on the same dedicated usage record completed batches use. Completes the runtime billing work in this stack: completed batches bill via #28360, interrupted batches via this PR.
Last of a four-PR stack splitting #28211. Stacked on #28361. Closes CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).
Problem
Interrupting a batch previously billed only the model window; work the tools already performed was lost.
Fix
Just before closing the buffer episode, the interrupt task reads the live tool stamps (
ToolCompletions) alongsideModelInvokedAt. The interruption commit appends the sharedbatchUsageMessagerecord (same shape as #28360: model-only tool-role row with abilled_ms/billed_callsaudit payload) billing the union of the started, billable calls' partial execution intervals: completed calls end at their recorded completion, running calls at the interrupt instant. Synthesized cancellation rows stay free of batch-level runtime. Unstarted calls, rejected calls, and sub-agent orchestration tools bill nothing and append no record. Batches with no live attempt (crash recovery, state promotion) bill nothing, consistent with generation losing in-flight runtime on a crash.Usage-record billing reuses the same interrupt instant as the model window instead of taking a second clock reading inside the transaction, so database delay cannot inflate a running call's window.
An earlier revision carried a retry-stable episode snapshot so an interrupt retry after buffer eviction (a >15s stall) would not lose billing; it was dropped as not worth the complexity (#28211 (comment)). Such a retry now bills the batch at zero, matching the existing crash-recovery precedent.
Known gaps left out of scope: retry backoff and errored-attempt time stay unbilled (no persistence home for error turns), and the advisor tool's nested model call now bills as tool time by design.