Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(coderd/x/chatd): bill interrupted tool batches on cancellation rows - #28362

Merged
jaaydenh merged 4 commits into
mainfrom
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches
Aug 25, 2026
Merged

feat(coderd/x/chatd): bill interrupted tool batches on cancellation rows#28362
jaaydenh merged 4 commits into
mainfrom
jaayden/codagt-928-stack-4-bill-interrupted-tool-batches

Conversation

@jaaydenh

@jaaydenh jaaydenh commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

Stack: #28359#28360#28361#28362 (this PR)

Summary

Interrupted local tool batches now bill the partial window they actually executed, recorded on the same dedicated usage record completed batches use. Completes the runtime billing work in this stack: completed batches bill via #28360, interrupted batches via this PR.

Last of a four-PR stack splitting #28211. Stacked on #28361. Closes CODAGT-928 (https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).

Problem

Interrupting a batch previously billed only the model window; work the tools already performed was lost.

Fix

Just before closing the buffer episode, the interrupt task reads the live tool stamps (ToolCompletions) alongside ModelInvokedAt. The interruption commit appends the shared batchUsageMessage record (same shape as #28360: model-only tool-role row with a billed_ms/billed_calls audit payload) billing the union of the started, billable calls' partial execution intervals: completed calls end at their recorded completion, running calls at the interrupt instant. Synthesized cancellation rows stay free of batch-level runtime. Unstarted calls, rejected calls, and sub-agent orchestration tools bill nothing and append no record. Batches with no live attempt (crash recovery, state promotion) bill nothing, consistent with generation losing in-flight runtime on a crash.

Usage-record billing reuses the same interrupt instant as the model window instead of taking a second clock reading inside the transaction, so database delay cannot inflate a running call's window.

An earlier revision carried a retry-stable episode snapshot so an interrupt retry after buffer eviction (a >15s stall) would not lose billing; it was dropped as not worth the complexity (#28211 (comment)). Such a retry now bills the batch at zero, matching the existing crash-recovery precedent.

Known gaps left out of scope: retry backoff and errored-attempt time stay unbilled (no persistence home for error turns), and the advisor tool's nested model call now bills as tool time by design.

@linear-code

linear-code Bot commented Aug 20, 2026

Copy link
Copy Markdown

CODAGT-928

@jaaydenh

Copy link
Copy Markdown
Contributor Author

@codex review

@jaaydenh
jaaydenh marked this pull request as ready for review August 20, 2026 11:50

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dea7b77830

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coder coder deleted a comment from chatgpt-codex-connector Bot Aug 20, 2026
@jaaydenh
jaaydenh force-pushed the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch from dea7b77 to a9a92e7 Compare August 20, 2026 13:05
@jaaydenh
jaaydenh force-pushed the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch from a9a92e7 to e9762eb Compare August 21, 2026 09:25
@jaaydenh
jaaydenh requested a review from hugodutka August 21, 2026 09:26
@jaaydenh
jaaydenh force-pushed the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch from e9762eb to c7f822d Compare August 22, 2026 07:06
@jaaydenh
jaaydenh force-pushed the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch from a306d1a to fb38522 Compare August 24, 2026 09:52
Comment thread coderd/x/chatd/ARCHITECTURE.md Outdated
Comment thread coderd/x/chatd/ARCHITECTURE.md Outdated
Comment thread coderd/x/chatd/tasks.go Outdated
jaaydenh added a commit that referenced this pull request Aug 25, 2026
…ocalTools (#28359)

**Stack**: **#28359 (this PR)** → #28360#28361#28362

`chatloop.ToolExecutionOutcome` only wrapped a `PersistedStep`, so
callers had to reach through `outcome.Step` for every field.
`ExecuteLocalTools` now returns the `PersistedStep` directly and the
wrapper type is deleted. No behavior change.

First of a four-PR stack that makes local tool execution count toward
Coder Agent runtime (splitting #28211). This refactor keeps the rename
noise out of the feature PRs.

Refs CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).
jaaydenh added a commit that referenced this pull request Aug 25, 2026
)

**Stack**: #28359 → **#28360 (this PR)** → #28361#28362

## Summary

Completed local tool batches now count toward Coder Agent runtime
(`chat_messages.runtime_ms`, the source of truth for
`hb_agent_runtime_v1`), excluding sub-agent orchestration tools to avoid
double counting, and without multiplying runtime for parallel tool
calls.

Second of a four-PR stack splitting #28211. Stacked on #28359. Refs
CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).

## Problem

Agent runtime previously measured only model invocation wall clock
(stream open to fully consumed). Time spent executing local tools
between steps, including file operations, terminal commands, workspace
provisioning, and MCP tools, was deliberately excluded, which
undercounts the product definition of "actively processing a task".

Naive inclusion has two hazards: a batch of parallel tool calls would
bill N windows for one wall-clock wait, and `wait_agent` would re-bill
child agents that already bill their own model and tool time.

## Fix

Each completed local tool batch bills one window: the union of the
billed tools' execution intervals, persisted as `runtime_ms` on a
dedicated usage record appended after the batch's tool-result rows. The
record is a tool-role message with `visibility='model'`, so it never
reaches the API, SSE, or clients, and prompt replay drops it because its
single internal `tool-batch-usage` part converts to no provider content.
Its content carries an audit payload (`billed_ms`, `billed_calls`) so
the billed window is inspectable after the fact; real tool-result rows
never carry batch-level runtime. Concurrent calls all start at batch
start, so 5 parallel 10s reads bill 10s, not 50s; serial calls
(`SerialToolCalls`) count only from their own launch, so unbilled waits
before them do not count.

Sub-agent orchestration tools (`spawn_agent`, `wait_agent`,
`message_agent`, `interrupt_agent`, `list_agents`,
`list_subagent_models`, plus the deprecated `close_agent` alias) never
extend the window: every chat, including children, applies the same
rules to its own runtime, so a parent's `wait_agent` window would double
count. A lone `wait_agent` bills 0 and appends no usage record;
`execute` 10s in parallel with `wait_agent` 60s bills 10s. A test pins
the unbilled set to the registered sub-agent tool catalog so they cannot
drift.

`GetTotalChatMessageRuntimeMsInRange` already sums `runtime_ms`
role-agnostically, so the usage record is picked up with no schema,
query, or cron changes.

Client-executed dynamic tools, external agents, parked/idle time, and
retry backoff remain unbilled. Interrupted batches still bill only the
model window; the rest of the stack adds partial-window billing via the
same usage record.

`ExecuteLocalToolsOptions` also gains an optional `ToolBillingRecorder`
(`RecordStart`/`RecordComplete`) observing per-occurrence dispatch-order
execution stamps. This PR wires no recorder; the next PR in the stack
connects it to the message part buffer for interrupt billing.

**NOTE**: Reported agent runtime (`hb_agent_runtime_v1`) increases from
deploy forward. There is no backfill and no feature flag.
@jaaydenh
jaaydenh force-pushed the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch 2 times, most recently from ea3720b to f6adb9f Compare August 25, 2026 09:56
Base automatically changed from jaayden/codagt-928-stack-3-buffer-tool-execution-stamps to main August 25, 2026 09:57
jaaydenh added a commit that referenced this pull request Aug 25, 2026
…e part buffer (#28361)

**Stack**: #28359#28360 → **#28361 (this PR)** → #28362

During local tool execution, buffer episodes now record when each
tool-call occurrence actually starts and finishes, via
`RecordToolStart`/`RecordToolCompletion`/`ToolCompletions` on the
message part buffer. Occurrences are keyed by unresolved-call position,
so gaps left by rejected calls and duplicate tool-call IDs stay
distinct: `partitionAmbiguousToolCalls` additionally returns the allowed
calls' input positions, and generation's `chatloop.ToolBillingRecorder`
implementation translates dispatch-order stamps back through them.

Mechanism only: nothing reads the stamps yet. The final PR in the stack
uses them to bill interrupted tool batches on a dedicated usage record
committed alongside their synthesized cancellation rows.

Third of a four-PR stack splitting #28211. Stacked on #28360. Refs
CODAGT-928
(https://linear.app/codercom/issue/CODAGT-928/track-local-tool-execution-for-agent-runtime).

---------

Co-authored-by: Hugo Dutka <[email protected]>
Interrupting a local tool batch previously billed only the model
window; work the tools already performed was lost. The interrupt task
now snapshots the episode's billing state (model invocation, live tool
stamps, buffered parts, and the interrupt instant) before its first
database read, and reuses that snapshot across task retries so buffer
eviction cannot lose it and retries cannot inflate it.

Synthesized cancellation rows bill the union of the started, billable
calls' partial execution intervals, placed once on the row whose
interval ends last. Unstarted, rejected, and sub-agent orchestration
calls bill nothing. The interrupted model window now also ends at the
snapshot instant instead of after the chat read.
… the buffer

The retry-stable interrupt snapshot guarded one rare case: an interrupt
task retrying after its episode was closed and garbage collected (a
database stall longer than the buffer's 15-second retention) would bill
nothing. That undercount matches the existing crash-recovery precedent
and is not worth the machinery, per review on the original PR.

StartInterrupt now reads ModelInvokedAt and ToolCompletions inline just
before closing the episode, and reuses the single interrupt instant for
cancellation-row billing so transaction delay cannot inflate windows.
…d usage record

Match the completed-batch billing shape: instead of stamping the
interval union on the cancellation row whose interval ends last, append
the shared batchUsageMessage record carrying the billed window and an
audit payload. Cancellation rows stay free of batch-level runtime.
@jaaydenh
jaaydenh force-pushed the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch from f6adb9f to 32e9e8e Compare August 25, 2026 09:57
@coderagents

coderagents Bot commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

Updates Needed

  • docs/ai-coder/agents/licensing-usage.md - The "Agent Time measurement" section no longer matches the billing behavior in this stack. The rewritten HBAgentRuntime doc comment in coderd/usage/usagetypes/events.go changes hb_agent_runtime_v1 so local tool execution time now counts toward Agent Time; the previous comment said local tool time "is excluded." The doc still describes the old model and needs updating:
    • The definition ("cumulative duration of model invocations") and the includes list (which mentions tool time only "before an interrupt") should state that local tool batch execution time now counts, billed as the union of execution intervals (parallel calls count once; serial calls count from their own start). This PR adds the interrupted-batch case on synthesized cancellation rows.
    • The excludes list should reflect the refined semantics: sub-agent orchestration and retry backoff bill nothing, and server-executed tools count even when their work runs in a connected workspace.

Automated review via Coder Agents

@jaaydenh
jaaydenh merged commit 511dcc0 into main Aug 25, 2026
28 checks passed
@jaaydenh
jaaydenh deleted the jaayden/codagt-928-stack-4-bill-interrupted-tool-batches branch August 25, 2026 10:29
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 25, 2026
@jaaydenh jaaydenh added cherry-pick/v2.37 Cherry-pick PR targeting release/2.37 cherry-pick and removed cherry-pick/v2.37 Cherry-pick PR targeting release/2.37 labels Aug 26, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants