diff --git a/docs/ai-coder/agents/licensing-usage.md b/docs/ai-coder/agents/licensing-usage.md index 047d4aaf137..035ba8e9d60 100644 --- a/docs/ai-coder/agents/licensing-usage.md +++ b/docs/ai-coder/agents/licensing-usage.md @@ -2,50 +2,114 @@ title: Licensing & Usage --- -Coder Agents is licensed differently depending on whether your deployment holds a Community license or an AI Premium license with a purchased Agent Time allocation. +Coder Agents licensing controls concurrent agent activity and Agent Time usage across your deployment. ## Community licenses Community licenses support up to five concurrently active agents deployment-wide. Coder doesn't limit how long those agents can run or how many tasks they complete over time. -Agents will queue when more than five agents are active at a time. +Agents queue when more than five agents are active at a time. With this agent pool, individuals and small teams can experiment with Coder Agents at no cost. ## AI Premium licenses and Agent Time AI Premium licenses include a customizable amount of Agent Time. Agent Time is shared across the deployment, allowing unlimited agents to run concurrently while consuming from a shared pool of purchased working hours. -This usage-based model is designed for enterprise workloads, where large development teams, background automation, and API-triggered tasks can create highly variable bursts of agent activity without being constrained by a concurrency limit. +This usage-based model supports enterprise workloads where large development teams, background automation, and API-triggered tasks can create variable bursts of agent activity. -## How Agent Time is measured +## Agent Time measurement -Agent Time is the cumulative duration during which an AI agent is actively processing a task for the user. -It is measured per interaction step and summed across the duration of a conversation. +Agent Time is the cumulative duration of model invocations that produce Coder Agents chat messages. +Coder measures each invocation from immediately before the request to the model provider opens until the response stream is fully consumed. -**Includes:** +Agent Time includes: -- Large language model inference time -- Tool execution (such as file operations, terminal commands, and workspace provisioning) -- Automated error recovery attempts +- Assistant generation steps in top-level and subagent chats. +- Context compaction (summarization) model calls. +- Model-provider tool execution within a model response. +- The time streamed or spent executing tools before an interrupt, kept on the partial assistant messages. -**Excludes:** +Agent Time excludes: -- Time the customer spends composing or reviewing messages -- Time between conversation turns when the agent is not processing -- Tool execution delegated to external systems outside the agent's direct control +- Time spent composing or reviewing messages. +- Time between conversation turns while an agent waits for user input. +- Time a parent agent spends waiting for a subagent, because the subagent records its own model invocations. +- Model calls that don't produce chat messages, such as title generation. +- Failed or retried model calls, which persist no content and record no runtime. +- Client-executed tools and chats handed off to external coding agents, since that work happens outside the server. -Agent Time does not accrue when a conversation is inactive or awaiting user input. -Agent Time is measured with millisecond precision and is rounded down to the nearest minute for billing purposes. +Coder records Agent Time in milliseconds and sums it across all Coder Agents chats in the deployment. ## Concurrency and usage limits -Coder handles concurrency and usage limits differently, depending on the license you use for Coder. +Coder handles concurrency and usage limits differently depending on the license you use. ### Community concurrency limit -When a Community license deployment reaches its limit of five concurrently active agents, any additional agents are placed in a queue. -As soon as an active agent completes its task, the next queued agent begins its work, so no work is lost and no action is required from the user. +When a Community license deployment reaches its limit of five concurrently active agents, Coder places additional agents in a queue. +When an active agent completes its task, the next queued agent begins its work. ### AI Premium Agent Time exhaustion -Coder sends deployment administrators an in-app soft warning message as the deployment approaches its maximum allotted Agent Time, so they can purchase additional Agent Time before the concurrency fallback takes effect. +Coder sends deployment administrators an in-app soft warning as the deployment approaches its maximum allotted Agent Time, so they can purchase additional Agent Time before the concurrency fallback takes effect. + +## Agent Time usage reporting + +Coder reports Agent Time usage to Tallyman, a Coder-managed server used for billing and reporting. +Coder sends the total Coder Agents runtime consumed per UTC hour, in milliseconds, with your deployment ID. +Coder doesn't send user-identifiable information or additional chat data to Tallyman. +Coder also shares the reported usage with [Metronome](https://metronome.com), a Stripe product and Coder partner for usage-based billing and reporting. + +Your Coder deployment must be able to make outbound HTTPS requests to `https://tallyman-prod.coder.com` to report usage. +Coder generates one `hb_agent_runtime_v1` event for each UTC hour shortly after the hour ends, then checks for unpublished events approximately every 17 minutes. +In steady state, each hour's usage typically reaches Tallyman within about 25 minutes of the hour closing. +You can monitor these requests in `coderd` logs. + +A successful request produces a debug log similar to the following example when you enable debug logging with [`CODER_LOG_FILTER=.*`](../../reference/cli/server.md#-l---log-filter): + +```sh +[debu] published usage events to tallyman accepted=1 rejected=0 +``` + +Coder sends the license JWT and deployment ID as request headers. +The request body contains one `hb_agent_runtime_v1` event for each UTC hour. +The `runtime_ms` value is the total Agent Time recorded during that hour. +Idle hours have a value of `0`. +If the deployment was offline, Coder backfills missed hours for up to seven days, batching up to 100 events per request. +Hours missing beyond seven days aren't reported. + +The following example reports 1 hour of Agent Time: + +```txt +POST /api/v1/events/ingest HTTP/1.1 +Host: tallyman-prod.coder.com +Content-Type: application/json +Coder-License-Key: +Coder-Deployment-ID: 8a4e92f1-3b7c-4d5e-9f12-abc123def456 + +{ + "events": [ + { + "id": "hb_agent_runtime_v1:2026-08-18_14:00:00", + "event_type": "hb_agent_runtime_v1", + "event_data": { + "runtime_ms": 3600000 + }, + "created_at": "2026-08-18T14:00:00Z" + } + ] +} +``` + +The event ID contains the start of the UTC hour. +The `created_at` value also identifies the start of that hour, rather than the time when Coder sends the request. +Coder sends raw milliseconds without rounding or converting the value to hours. + +A failed request produces a warning similar to the following example: + +```sh +[warn] failed to send publish request to tallyman count=1 error="Post \"https://tallyman-prod.coder.com/api/v1/events/ingest\": dial tcp: lookup tallyman-prod.coder.com: no such host" +``` + +> [!NOTE] +> Air-gapped deployments and deployments with legal restrictions around usage reporting can [contact us](https://coder.com/contact) to discuss alternative methods. diff --git a/docs/ai-coder/ai-governance.md b/docs/ai-coder/ai-governance.md index d047288b41d..e4b56c04b77 100644 --- a/docs/ai-coder/ai-governance.md +++ b/docs/ai-coder/ai-governance.md @@ -42,33 +42,33 @@ or without Coder's AI Governance features. Organizations adopting AI coding tools at scale often encounter operational and security challenges that traditional developer tooling doesn't address. -### Auditing AI activity across teams +### Audit AI activity across teams Without centralized monitoring, teams have no way to understand how AI tools are being used across the organization. AI Gateway provides audit trails of prompts, token usage, and tool invocations, giving administrators insight into AI adoption patterns and potential issues. -### Restricting agent network access +### Restrict agent network access AI agents can make arbitrary network requests, potentially accessing unauthorized services or exfiltrating data. Agent Firewall enforces process-level policies that restrict which domains agents can reach and what actions they can perform, preventing unintended data exposure. -### Centralizing API key management +### Centralize API key management Managing individual API keys for AI providers across hundreds of developers creates security risks and administrative overhead. AI Gateway centralizes authentication so users authenticate through Coder, eliminating the need to distribute and rotate provider API keys. -### Standardizing MCP tools and servers +### Standardize MCP tools and servers Different teams may use different MCP servers and tools with varying security postures. AI Gateway enables centralized MCP administration, allowing organizations to define approved tools and servers that all users can access. -### Measuring AI adoption and spend +### Measure AI adoption and spend Without usage data, it's hard to justify AI tooling investments or identify high-leverage use cases. AI Gateway captures metrics on token spend, adoption @@ -81,65 +81,3 @@ generally available as part of AI Governance. To learn more about AI Governance, pricing, or trial options, reach out to your [Coder account team](https://coder.com/contact/sales). - -## How Coder Tasks usage is measured - -> [!NOTE] -> There is a known issue with how Agent Workspace Builds are tallied in v2.28 -> and v2.29. We recommend updating to v2.28.9, v2.29.4, or v2.30 to resolve -> this issue. - -The usage metric used to measure Coder Tasks consumption is called **Agent -Workspace Builds** (prev. "managed agents"). - -An Agent Workspace Build is counted each time a workspace is started -specifically for a coding agent to independently work on a Coder Task. Most of -the work in this workspace is performed by the agent, not a human developer. -Each Coder Task starts its own workspace, and the usage meter counts one Agent -Workspace Build. - -Traditional Coder Workspaces started manually by developers or scheduled to -auto-start do not count as an Agent Workspace Build. These are considered -daily-driver development environments where developers co-exist with their IDEs -and coding assistants. - -### Scenarios - -| Scenario | Consumes Agent Workspace Build | -|---------------------------------------------------------------------------------------------------|--------------------------------| -| Developer creates a Coder Task to write end-to-end tests | Yes | -| Automated pipeline creates a task via Coder Tasks CLI (with Claude Code) to review a pull request | Yes | -| Developer resumes an old Coder Task order to continue prototyping | Yes | -| Developer starts a workspace for use with VS Code and Jupyter | No | -| Developer creates a workspace for use with Cursor and Claude Code CLI | No | -| Developer creates a workspace for use with Coder AI Gateway and Agent Firewall | No | - -In the future, additional capabilities for managing agents (beyond Coder Tasks) -may also consume agent workspace builds. - -### Agent Workspace Build Limits - -Without proper controls and sandboxing, it is not recommended to open up Coder -Tasks to a large audience in the enterprise. Both Community and Premium -deployments include 1,000 Agent Workspace Builds, primarily for proof-of-concept -use and basic workflows. Community deployments do not have access to -[AI Gateway](./ai-gateway/index.md) or [Agent Firewall](./agent-firewall/index.md). - -Premium deployments include a shared usage pool of Agent Workspace Builds for -automated workflows, along with limits that scale proportionately with user -count. Usage counts are measured and sent to Coder via -[usage data reporting](./usage-data-reporting.md). Coder Tasks and other AI -features continue to function normally even if the limit is breached. Admins -will receive a warning to [contact their account team](https://coder.com/contact) -to remediate. - -### Tracking Agent Workspace Builds - -Admins can monitor Agent Workspace Build usage from the Coder dashboard. -Navigate to **Deployment** > **Licenses** to view current usage against your -entitlement limits. - -![Agent Workspace Build usage](../images/admin/ai-governance-awb-usage.png) - -Agent Workspace Build usage showing current consumption against -entitlement limits in the Licenses page. diff --git a/docs/ai-coder/index.md b/docs/ai-coder/index.md index 0381687a3ce..541cf536844 100644 --- a/docs/ai-coder/index.md +++ b/docs/ai-coder/index.md @@ -48,8 +48,5 @@ Coder deployment. It includes: - [Agent Firewall](./agent-firewall/index.md) for process-level network and command policies that restrict what agents can reach and do inside a workspace. -- Expanded Agent Workspace Build allowances for teams running AI-driven - background work at scale. - [Learn more about AI Governance](./ai-governance.md) for use cases, entitlements, and how to enable it in your deployment. diff --git a/docs/ai-coder/usage-data-reporting.md b/docs/ai-coder/usage-data-reporting.md deleted file mode 100644 index 3c69e27ae0b..00000000000 --- a/docs/ai-coder/usage-data-reporting.md +++ /dev/null @@ -1,84 +0,0 @@ -# Usage Data Reporting - -[AI Governance](./ai-governance.md) requires reporting usage data to Tallyman, a Coder-managed server for billing and reporting purposes. Coder only captures and sends the following information, related to your deployment ID: - -- number of agent workspace builds consumed -- number of AI Governance seats consumed -- total Coder Agent runtime consumed per hour, in milliseconds - -No user-identifiable information or additional metrics are sent to Tallyman. This information is also shared with [Metronome](https://metronome.com), a Stripe product and Coder partner for usage-based billing and reporting. - -To send usage data, your Coder deployment must be able to make outbound HTTPS requests to `https://tallyman-prod.coder.com`. Usage data is sent approximately every 17 minutes and can be monitored via `coderd` logs. - -Example of a successful request (requires debug logging enabled [`CODER_LOG_FILTER=.*`](../reference/cli/server.md#-l---log-filter)): - -```sh -[debu] published usage events to tallyman accepted=5 rejected=0 -``` - -Example of a request payload: - -```txt -POST /api/v1/events/ingest HTTP/1.1 -Host: tallyman-prod.coder.com -Content-Type: application/json -Coder-License-Key: # your license JWT for verification -Coder-Deployment-ID: 8a4e92f1-3b7c-4d5e-9f12-abc123def456 # your deployment ID - -{ - "events": [ - { - "id": "550e8400-e29b-41d4-a716-446655440000", # unique event ID generated by Coder - "event_type": "dc_managed_agents_v1", # aka. agent workspace builds - "event_data": { - "count": 1 - }, - "created_at": "2025-01-15T14:30:00Z" - }, - { - "id": "hb_agent_runtime_v1:2025-01-15_14:00:00", # unique event ID generated by Coder - "event_type": "hb_agent_runtime_v1", # aka. Coder Agent runtime - "event_data": { - "runtime_ms": 3600000 - }, - "created_at": "2025-01-15T14:00:00Z" # start of the hour the runtime was consumed in - } - ] -} -``` - -Example of a failed request (e.g. Tallyman Server is blocked by your network): - -```sh -[warn] failed to send publish request to tallyman count=5 error="Post \"https://tallyman-prod.coder.com/api/v1/events/ingest\": dial tcp: lookup tallyman-prod.coder.com: no such host" -``` - -> [!NOTE] -> Air-gapped deployments and/or those with legal restrictions around usage reporting can [contact us](https://coder.com/contact) to discuss alternative methods. - -## Agent runtime measurement - -Total Coder Agent runtime is summed from per-message generation time -(`runtime_ms` on chat messages). - -A message's runtime is the wall-clock duration of the model invocation that -produced its content, measured from just before the request to the model -provider opens until the response is fully consumed. - -What counts: - -- Assistant generation steps, in both top-level chats and sub-agent chats. -- Context compaction (summarization) model calls. -- Interrupted generation: the time streamed before the interrupt is kept on - the partial assistant message. - -What does not count: - -- Local tool execution, including waiting on sub-agents. A sub-agent is its - own chat and records its own model invocations, so counting the parent's - wait would double count. -- Idle time: chats waiting for user input or external tool results. -- Failed model calls whose output was discarded. Retried and errored - attempts persist no content, so they record no runtime. -- Ancillary model calls that produce no chat messages, such as title - generation. diff --git a/docs/images/admin/ai-governance-awb-usage.png b/docs/images/admin/ai-governance-awb-usage.png deleted file mode 100644 index 48e18583086..00000000000 Binary files a/docs/images/admin/ai-governance-awb-usage.png and /dev/null differ diff --git a/docs/manifest.json b/docs/manifest.json index 9e5c1977ce7..91e85c173d3 100644 --- a/docs/manifest.json +++ b/docs/manifest.json @@ -1126,7 +1126,7 @@ }, { "title": "Licensing \u0026 Usage", - "description": "Licensing and usage details for Coder Agents.", + "description": "Compare Coder Agents licenses and learn how Coder measures and reports Agent Time usage.", "path": "./ai-coder/agents/licensing-usage.md" }, { @@ -1342,12 +1342,6 @@ "state": ["premium"] } ] - }, - { - "title": "Usage Data Reporting", - "description": "Learn what AI usage data AI Governance reports to Coder for billing and reporting.", - "path": "./ai-coder/usage-data-reporting.md", - "state": ["premium"] } ] }, diff --git a/docs/reference/glossary.md b/docs/reference/glossary.md index 2dc29929f34..3305e70602b 100644 --- a/docs/reference/glossary.md +++ b/docs/reference/glossary.md @@ -24,12 +24,6 @@ It was previously named Agent Boundaries and uses a sandbox backend, `nsjail` by This feature requires a Premium license. Refer to [Agent Firewall](../ai-coder/agent-firewall/index.md). -### Agent Workspace Build - -A metered workspace build performed on behalf of an AI agent. -Community and Premium deployments include 1,000 for proof-of-concept use, and a Premium license expands the allowance. -Refer to [AI Governance](../ai-coder/ai-governance.md). - ### AI Gateway An LLM gateway in `coderd` that authenticates users, forwards traffic to providers such as OpenAI and Anthropic, audits prompts and tool invocations, and centralizes MCP administration. @@ -45,7 +39,7 @@ Refer to [AI Gateway Proxy](../ai-coder/ai-gateway/ai-gateway-proxy/index.md). ### AI Governance -Included with a Premium license, AI Governance unlocks AI Gateway and Agent Firewall and expands Agent Workspace Build allowances. +Included with a Premium license, AI Governance unlocks AI Gateway and Agent Firewall. Refer to [AI Governance](../ai-coder/ai-governance.md). ### Air-gapped deployment @@ -126,12 +120,6 @@ Refer to [Coder Desktop](../user-guides/desktop/index.md). The editor extension that connects VS Code, and forks such as Cursor and Devin Desktop (formerly Windsurf), to Coder workspaces. Refer to [VS Code](../user-guides/workspace-access/vscode.md). -### Coder Tasks - -An earlier interface for running coding agents such as Claude Code and Aider inside workspaces. -Coder Tasks is deprecated: it moves to a 12-month Extended Support Release for Premium customers and is removed from new releases starting with v2.37, with [Coder Agents](#coder-agents) as the long-term replacement. -Refer to [Coder Tasks](../ai-coder/tasks.md). - ### `coder_agent` The Terraform resource that declares a [workspace agent](#workspace-agent) inside a template. @@ -165,7 +153,7 @@ Refer to the [`codersdk` package](https://pkg.go.dev/github.com/coder/coder/v2/c ### Coding agent -An AI agent that reads and writes code on a developer's behalf, such as Claude Code, run through Coder Tasks or Coder Agents. +An AI agent that reads and writes code on a developer's behalf, such as Claude Code or Coder Agents. Refer to [AI in Coder](../ai-coder/index.md). ### Community