fix: register ai-seats usage cron and grant usage-publisher create permission - #27508
fix: register ai-seats usage cron and grant usage-publisher create permission#27508jaaydenh wants to merge 2 commits into
Conversation
…rmission The ai-seats usage cron never ran: its CronJob was registered without the required EventType, so Register rejected it, and the error was discarded. ai-seats heartbeats drive AI addon seat consumption, so no seat usage events were being generated. Registering the job also surfaces a second latent bug: the usage-publisher subject lacked usage_event create permission, so every heartbeat insert performed under AsUsagePublisher would fail authz. Grant the permission and add a strict-authorizer regression test for the insert path.
|
/coder-agents-review |
|
@codex review |
|
Review starting... deep-review v0.9.0 | Round 1 | Last posted: Round 1, 1 findings (1 Note), COMMENT. Review Finding inventoryFinding inventory: PR #27508Findings
Round logRound 1Panel of 16 (Bisky, Hisoka, Mafu-san, Mafuuu, Pariston, Kurapika, Razor, ging-go, Chopper, Gon, Leorio, Komugi, Kite, Ryosuke, Meruem, Zoro) plus Netero first pass. No Law (38 effective additions). Reviewed against 6120fb5..fa69e39. Unanimous verdict: the change is correct, minimal, and verified. All 16 panel reviewers plus Netero returned "No findings" at P-level. Multiple reviewers independently reverted the ActionCreate grant and confirmed TestUsagePublisherCanInsertHeartbeats fails with a named authz error, proving the regression test is genuine. Severity spread: 0 P0-P4, 1 Nit (commit subject length), 5 Notes (4 distinct + coverage-boundary consensus). Orchestrator verified the coverage-boundary claim: usageCron.Register is unconditional at enterprise boot and TestServer_Single exercises it, so the fixed regression class (missing/invalid EventType) is mechanically caught at startup. CRF-1 captures the residual gap (wrong-but-valid heartbeat type reconciled only at runtime). About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
|
Codex Review: Didn't find any major issues. Keep them coming! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
The ai-seats cron registration fix now ships independently in #27508 so it is not coupled to this feature's Tallyman release gate. The usage-publisher create permission stays here because the generator's heartbeat inserts also require it; both PRs carry the identical hunk.
…reate comment Uses the same comment text as the agent-runtime feature branch so the overlapping permission hunk merges cleanly whichever PR lands second.
|
/coder-agents-review |
|
Chat: Review posted | View chat Review history
|
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Clean, well-scoped bug fix. The diagnosis is accurate and the fix lands at the root cause on both halves: registering the ai-seats job with its required EventType (and no longer discarding the Register error with _ =), and granting subjectUsagePublisher the usage_event create permission the heartbeat insert actually needs. The strict-authorizer regression test is genuine, not decorative: multiple reviewers independently reverted the ActionCreate grant and confirmed it fails with a named forbidden ... (action: create), (object: usage_event) authz error plus an unmet InsertUsageEvent mock expectation. 63% test density is proportional for a two-part fix, and the error-cleanup path matches the adjacent publisher.Start() convention (no leak: usageCron is not yet in closers and Start has not run).
A 16-reviewer panel plus a mechanical first pass all returned no P-level findings. Severity spread: 0 P0-P4, 1 Nit, a few Notes.
As Bisky put it: "Delete the feature and the test goes red. Genuine gem."
One inline Note below (CRF-1). The remaining observations are non-blocking and collected here:
Coverage boundary (Bisky, Mafu-san, Mafuuu, Chopper, Meruem, Pariston converged). No test directly asserts that server.go registers ai-seats with a valid heartbeat EventType. This is acceptable and mechanically covered for the exact class this PR fixes: the Register call is unconditional at enterprise boot, and because its error is now propagated, a reintroduced missing or non-heartbeat EventType aborts startup. TestServer_Single boots the full enterprise server (no license required) and waits on /healthz, so that recurrence would fail CI. The residual gap (a wrong-but-still-heartbeat type, reconciled only by the runtime log-and-continue check in cron.run) is captured in CRF-1. No change needed now.
Nit (Leorio, CRF-2). Both commit subjects overflow 72 chars (82 and 77), so git log --oneline, the PR list, and blame views truncate them where the useful noun sits. The scope path and imperative mood are correct; only the length. Suggested trims: fix: register ai-seats usage cron, grant usage-publisher create perm (68) and chore(coderd/database/dbauthz): neutral usage-publisher create comment (69).
Note (Pariston, Ryosuke, CRF-3). The usage_event create hunk in dbauthz.go is identical to the one in #27312, so the two land in either order without conflict as the description states. Flagging only so a future reviewer does not read the duplication as a rebase artifact, and so a later divergence between the two copies is not silent.
Note (Netero, CRF-4). cron_test.go:67 carries a pre-existing em-dash (U+2014) in a comment. It is out of this diff and out of CI's diff-scoped em-dash check, but since the author is already editing this file, fixing it here is cheap.
🤖 This review was automatically generated with Coder Agents.
| err = usageCron.Register(usage.CronJob{ | ||
| Name: "ai-seats", | ||
| Interval: usage.AISeatsInterval, | ||
| EventType: usagetypes.UsageEventTypeHBAISeatsV1, |
There was a problem hiding this comment.
Note [CRF-1] CronJob.EventType duplicates the type Fn already produces, and the two are reconciled only at runtime. (Meruem, Ryosuke)
Register validates job.EventType.IsHeartbeat() but cannot check that EventType matches what Fn returns, because Fn hits the database and cannot run at registration time. The match is instead verified inside cron.run via event.EventType() != job.EventType, where a mismatch logs an error and continues. That is the same failure class this PR fixes (a job that is configured but emits zero usage events), relocated from registration to execution: a future job whose EventType disagrees with its Fn would silently stop billing events with only a warning line.
Correct today: ai-seats is the only registered job, EventType matches AISeatsHeartbeat's output, and TestAISeatsHeartbeat pins that output. No change requested for this PR. If a second job is ever added, a table-driven test asserting each registered job's EventType equals its Fn's produced type would make the invariant mechanical rather than trusting the runtime log.
🤖
…time (#27312) closes CODAGT-839 closes CODAGT-843 closes CODAGT-773 ## Summary Adds a new heartbeat usage event type, `hb_agent_runtime_v1`, measuring the total agent-loop runtime of Coder Agents (chats) per UTC hour, plus a reconciler that generates one event per hour with self-healing backfill over a trailing 7-day window. Events flow to Tallyman through the existing publisher unchanged. This measures the new Coder Agents (the `chats` tables), not the deprecated Tasks counted by `dc_managed_agents_v1`. Independent of #27508, which fixes the dead ai-seats cron registration. Both PRs carry the identical `usage_event` create permission hunk for the usage-publisher subject (this feature's generator and the ai-seats cron each need it for heartbeat inserts), so they can land in either order and the overlap merges cleanly. > [!WARNING] > **Do not include this in a release until Tallyman accepts `hb_agent_runtime_v1`.** The publisher marks permanently rejected events as done-forever, and the generator then sees those buckets as complete locally, so their usage would be silently and permanently lost. ## Details Each event's payload is `{"runtime_ms": N}`: the sum of `chat_messages.runtime_ms` for messages created in the hour bucket `[H, H+1)`, across all chats (sub-agents, API-created, archived, and soft-deleted messages included). Events use deterministic IDs (`hb_agent_runtime_v1:<bucket start>`) with `created_at` set to the bucket start, so concurrent replicas race safely via `ON CONFLICT (id) DO NOTHING` without locking, and daily rollups attribute backfilled hours to the correct day. Idle hours produce zero-valued events. A bucket becomes eligible 5 minutes after it closes; hours missing for longer than the 7-day window are forfeited, which can only undercount. Note that this makes `usage_events.created_at` explicitly the *event occurrence time* rather than the row insertion time; the two only diverge for backfilled events. It already behaved as the occurrence timestamp (it drives the daily rollup day and is shipped to Tallyman/Metronome as the event timestamp), and the migration now documents this with a `COMMENT ON COLUMN`, which also surfaces as a Go doc comment on `UsageEvent.CreatedAt`. The new `usage.Generator` runs unconditionally in enterprise builds; the `publish_usage_data` license flag continues to gate egress only, so air-gapped deployments still fill their local ledger. The `aggregate_usage_event()` trigger sums `runtime_ms` per day into `usage_events_daily` (unlike `hb_ai_seats_v1`, which takes the daily max). `InsertHeartbeatUsageEvent` now takes an explicit `createdAt` so generators can backfill historical buckets; the cron passes `clock.Now()` to preserve its existing behavior. ## Tallyman follow-up <details> <summary>Prompt for the Tallyman-repo agent</summary> > **Task**: Add support for the new Coder usage event type `hb_agent_runtime_v1` so Tallyman accepts, validates, and forwards it to Metronome. > > **Background**: coder/coder PR (this PR) adds hourly heartbeat events measuring Coder Agent runtime. Events arrive via the existing `/api/v1/events/ingest` endpoint with: `event_type: "hb_agent_runtime_v1"`, `event_data: {"runtime_ms": <int64 >= 0>}`, deterministic `id` of the form `hb_agent_runtime_v1:2026-07-15_14:00:00` (UTC hour bucket start), and `created_at` set to the bucket start (may be up to ~8 days in the past due to backfill; within Metronome's 34-day dedup window). Zero-value events are normal (idle hours). > > **Work**: > 1. Update Tallyman's vendored/imported `coderd/usage/usagetypes` (or equivalent) to the coder/coder commit that adds `UsageEventTypeHBAgentRuntimeV1` and `HBAgentRuntime`. > 2. Ensure ingestion validation accepts the type (`Valid()` switches) and rejects negative `runtime_ms`. > 3. Ensure Metronome forwarding maps the event with transaction ID derived from the event `id` as for existing types, passing `runtime_ms` through as the property for a SUM-aggregated billable metric ("Coder Agent Hours" = `SUM(runtime_ms) / 3,600,000`). > 4. Do NOT permanently reject unknown-but-well-formed future `hb_*` types if avoidable; at minimum confirm current behavior for unknown types (temporary vs permanent rejection) and report it. > 5. Tests: ingest accept/validate, dedup by ID, Metronome payload mapping. > > **Constraint**: this must be deployed to tallyman-prod **before** any coder/coder release containing the event generator; coderd treats permanent rejections as terminal per event. </details>
The ai-seats usage cron never ran on main: its
CronJobwas registered without the requiredEventType, soRegisterrejected it ("event type must be a heartbeat type"), and the error was discarded with_ =, leaving the job out of the cron's job list entirely. ai-seats heartbeats drive AI addon seat consumption on the License, so no seat usage events are being generated today.Registering the job also surfaces a second latent bug:
subjectUsagePublisherlackedusage_eventcreate permission (its doc comment already claimed create), so every heartbeat insert performed underAsUsagePublisherwould fail authz the moment the cron actually ran. This PR grants the permission and adds a strict-authorizer regression test covering the insert path.Split out of #27312 per review feedback so this unblocked billing fix is not coupled to that feature's external Tallyman release gate. The two PRs are independent: both carry the identical usage-publisher create permission hunk, so they can land in either order.