Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: normalize workspace agent session counts into a child table - #27952

Closed
EhabY wants to merge 2 commits into
mainfrom
feat/normalized-session-counts
Closed

feat: normalize workspace agent session counts into a child table#27952
EhabY wants to merge 2 commits into
mainfrom
feat/normalized-session-counts

Conversation

@EhabY

@EhabY EhabY commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

Supersedes #27179 (same branch, squashed and rebased).

Phase 1 of the Scalable Approach for Adding New IDE Session Types RFC, tracked in #27410.

Summary

Replaces the four fixed session_count_* columns on workspace_agent_stats with a sparse session_counts JSONB object keyed by normalized app name. New IDEs can report session counts without schema changes, while existing APIs and metrics continue to expose the four well-known families.

Problem

Session counts were pinned to four hardcoded columns (vscode, jetbrains, reconnecting_pty, and ssh). Every new IDE was mislabeled or dropped, and adding a name required changes across the schema and application.

A normalized child-table implementation was benchmarked at up to 10 million rows and caused substantial read, write, storage, WAL, and migration regressions. Keeping the normalized object on the parent row avoids the extra relation and join overhead.

Fix

  • Agent API v2.11 adds a session_counts map to Stats, deprecating fields 8 to 11. The server converts the fixed fields only when the map is empty.
  • coderd/idemetadata holds the shared vocabulary. App names are normalized at ingestion, reports are capped at 64 distinct names, and overflow is combined into unknown.
  • workspace_agent_stats.session_counts stores one sparse JSONB object per stats row. Arbitrary keys are preserved, while existing read paths extract the four well-known names.
  • Agent counters are created on demand. POST /workspaces/{workspace}/usage and coder ssh --usage-app accept arbitrary app names.
  • Usage queries process the latest completed minute without including rows before the requested cutoff, and template insights deduplicate activity by template, user, and minute.
  • Removes agentsdk.Stats, which has been unreachable since the HTTP stats endpoint was removed.

Migration requirement

NOTE: Migration 569 takes an ACCESS EXCLUSIVE lock on workspace_agent_stats, backfills the retained raw-stat window, rebuilds the template-insights covering index, and drops the four fixed columns.

Deployments with retained rows containing positive session activity must have a template_usage_stats rollup from within the previous 24 hours. If the watermark is missing or stale, the migration aborts before changing the schema. Run the previous Coder version until template usage stats roll up, then retry the upgrade. Check coderd logs for failed to rollup data if the watermark does not advance.

Idle-only databases can upgrade without a watermark because their rows already match the {} default. Databases containing only raw stats older than the 180-day retention fallback can also upgrade.

With a healthy watermark, the 10 million-row benchmark converted approximately 333,000 rows in 4.44 seconds on PostgreSQL 17 tmpfs and 6.13 seconds on durable storage. Reads and writes remain blocked for most of the migration, and runtime scales with recent ingestion volume.

Mixed-version note: a new CLI passing a custom --usage-app value to an older coderd gets a 400 because old servers still validate against the fixed allowlist.

Follow-ups: #27411 (expose per-app counts in stats, Prometheus, UI), #27412 (connection_logs.type ENUM to TEXT), #27413 (template_usage_stats *_mins to JSONB), coder/vscode-coder#1044 (report the real IDE from the extension).

PR description updated by Coder Agents.

@EhabY
EhabY force-pushed the feat/normalized-session-counts branch 3 times, most recently from ac127f6 to ead98bf Compare August 11, 2026 13:32
@EhabY
EhabY force-pushed the feat/normalized-session-counts branch from ead98bf to e8ce184 Compare August 11, 2026 13:52
@EhabY
EhabY requested a review from code-asher August 11, 2026 13:53
Replace the fixed session_count_* columns on workspace_agent_stats with a
workspace_agent_session_counts child table keyed by app name, so any IDE
can report session counts without schema changes. Agent API v2.11 adds a
session_counts map to Stats, deprecating the fixed fields.
@EhabY
EhabY force-pushed the feat/normalized-session-counts branch from e8ce184 to 07b995b Compare August 11, 2026 14:02
@EhabY
EhabY marked this pull request as ready for review August 11, 2026 14:32
@coderagents

coderagents Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

Updates Needed

  • docs/reference/api/schemas.md (codersdk.UsageAppName, referenced by POST /api/v2/workspaces/{workspace}/usage) - This PR removes the server-side allowlist validation on app_name (codersdk.AllowedAppNames and the 400 in coderd/workspaces.go), so the endpoint now accepts arbitrary app names, normalized at ingestion. The published API reference still advertises a closed enum (vscode, jetbrains, reconnecting-pty, ssh), which now under-represents the accepted values. This page is auto-generated, so do not hand-edit it. Instead relax or annotate the swagger source, the UsageAppName type in codersdk/workspaces.go, so the regenerated reference reflects that any app name is accepted (well-known names normalized on the server). Then run make gen.

Automated review via Coder Agents

@spikecurtis spikecurtis left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@EhabY this needs to be split into smaller PRs. It's too big for any human to give a high quality review.

@EhabY

EhabY commented Aug 13, 2026

Copy link
Copy Markdown
Contributor Author

Closing this in favor of this stack: #28124 (first PR)

@EhabY EhabY closed this Aug 13, 2026
@github-actions github-actions Bot locked and limited conversation to collaborators Aug 13, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants