Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix: register ai-seats usage cron and grant usage-publisher create permission - #27508

Closed
jaaydenh wants to merge 2 commits into
mainfrom
fix-ai-seats-cron-registration
Closed

fix: register ai-seats usage cron and grant usage-publisher create permission#27508
jaaydenh wants to merge 2 commits into
mainfrom
fix-ai-seats-cron-registration

Conversation

@jaaydenh

@jaaydenh jaaydenh commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

The ai-seats usage cron never ran on main: its CronJob was registered without the required EventType, so Register rejected it ("event type must be a heartbeat type"), and the error was discarded with _ =, leaving the job out of the cron's job list entirely. ai-seats heartbeats drive AI addon seat consumption on the License, so no seat usage events are being generated today.

Registering the job also surfaces a second latent bug: subjectUsagePublisher lacked usage_event create permission (its doc comment already claimed create), so every heartbeat insert performed under AsUsagePublisher would fail authz the moment the cron actually ran. This PR grants the permission and adds a strict-authorizer regression test covering the insert path.

Split out of #27312 per review feedback so this unblocked billing fix is not coupled to that feature's external Tallyman release gate. The two PRs are independent: both carry the identical usage-publisher create permission hunk, so they can land in either order.

…rmission

The ai-seats usage cron never ran: its CronJob was registered without
the required EventType, so Register rejected it, and the error was
discarded. ai-seats heartbeats drive AI addon seat consumption, so no
seat usage events were being generated.

Registering the job also surfaces a second latent bug: the
usage-publisher subject lacked usage_event create permission, so every
heartbeat insert performed under AsUsagePublisher would fail authz.
Grant the permission and add a strict-authorizer regression test for
the insert path.

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

Copy link
Copy Markdown
Contributor Author

@codex review

@coder-agents-review

coder-agents-review Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Review starting...
Requested: 2026-07-27 08:40 UTC by @jaaydenh

deep-review v0.9.0 | Round 1 | 6120fb5..fa69e39

Last posted: Round 1, 1 findings (1 Note), COMMENT. Review

Finding inventory

Finding inventory: PR #27508

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 Note Open enterprise/cli/server.go:152 CronJob.EventType duplicates Fn's output, reconciled only at runtime (log-and-continue); structural debt if a second job is added R1 Meruem, Ryosuke Yes
CRF-2 Nit Open git commit subjects Both commit subjects exceed 72 chars (82, 77) R1 Leorio No (body)
CRF-3 Note Open coderd/database/dbauthz/dbauthz.go:669 usage_event create hunk is identical to #27312; silent divergence risk if the copies ever drift R1 Pariston, Ryosuke No (body)
CRF-4 Note Open enterprise/coderd/usage/cron_test.go:67 Pre-existing em-dash (U+2014) in a comment, out of diff but in an edited file; cheap to fix while here R1 Netero No (body)
CRF-5 Note Open enterprise/cli/server.go:149 server.go registration wiring has no direct regression test; mitigated by now-fatal Register error caught at boot by TestServer_Single R1 Bisky, Mafu-san, Mafuuu, Chopper, Meruem, Pariston No (body)

Round log

Round 1

Panel of 16 (Bisky, Hisoka, Mafu-san, Mafuuu, Pariston, Kurapika, Razor, ging-go, Chopper, Gon, Leorio, Komugi, Kite, Ryosuke, Meruem, Zoro) plus Netero first pass. No Law (38 effective additions). Reviewed against 6120fb5..fa69e39.

Unanimous verdict: the change is correct, minimal, and verified. All 16 panel reviewers plus Netero returned "No findings" at P-level. Multiple reviewers independently reverted the ActionCreate grant and confirmed TestUsagePublisherCanInsertHeartbeats fails with a named authz error, proving the regression test is genuine.

Severity spread: 0 P0-P4, 1 Nit (commit subject length), 5 Notes (4 distinct + coverage-boundary consensus). Orchestrator verified the coverage-boundary claim: usageCron.Register is unconditional at enterprise boot and TestServer_Single exercises it, so the fixed regression class (missing/invalid EventType) is mechanically caught at startup. CRF-1 captures the residual gap (wrong-but-valid heartbeat type reconciled only at runtime).

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep them coming!

Reviewed commit: b3eacc76c3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

jaaydenh added a commit that referenced this pull request Jul 27, 2026
The ai-seats cron registration fix now ships independently in #27508 so
it is not coupled to this feature's Tallyman release gate. The
usage-publisher create permission stays here because the generator's
heartbeat inserts also require it; both PRs carry the identical hunk.
…reate comment

Uses the same comment text as the agent-runtime feature branch so the
overlapping permission hunk merges cleanly whichever PR lands second.

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review

coder-agents-review Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-07-27 09:11 UTC by @jaaydenh
Spend: $14.41 / $100.00

Review history
  • R1 (2026-07-27): 16 reviewers, 1 Note, COMMENT. Review

🤖 Managed by Coder Agents.

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: fa69e39820

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean, well-scoped bug fix. The diagnosis is accurate and the fix lands at the root cause on both halves: registering the ai-seats job with its required EventType (and no longer discarding the Register error with _ =), and granting subjectUsagePublisher the usage_event create permission the heartbeat insert actually needs. The strict-authorizer regression test is genuine, not decorative: multiple reviewers independently reverted the ActionCreate grant and confirmed it fails with a named forbidden ... (action: create), (object: usage_event) authz error plus an unmet InsertUsageEvent mock expectation. 63% test density is proportional for a two-part fix, and the error-cleanup path matches the adjacent publisher.Start() convention (no leak: usageCron is not yet in closers and Start has not run).

A 16-reviewer panel plus a mechanical first pass all returned no P-level findings. Severity spread: 0 P0-P4, 1 Nit, a few Notes.

As Bisky put it: "Delete the feature and the test goes red. Genuine gem."

One inline Note below (CRF-1). The remaining observations are non-blocking and collected here:

Coverage boundary (Bisky, Mafu-san, Mafuuu, Chopper, Meruem, Pariston converged). No test directly asserts that server.go registers ai-seats with a valid heartbeat EventType. This is acceptable and mechanically covered for the exact class this PR fixes: the Register call is unconditional at enterprise boot, and because its error is now propagated, a reintroduced missing or non-heartbeat EventType aborts startup. TestServer_Single boots the full enterprise server (no license required) and waits on /healthz, so that recurrence would fail CI. The residual gap (a wrong-but-still-heartbeat type, reconciled only by the runtime log-and-continue check in cron.run) is captured in CRF-1. No change needed now.

Nit (Leorio, CRF-2). Both commit subjects overflow 72 chars (82 and 77), so git log --oneline, the PR list, and blame views truncate them where the useful noun sits. The scope path and imperative mood are correct; only the length. Suggested trims: fix: register ai-seats usage cron, grant usage-publisher create perm (68) and chore(coderd/database/dbauthz): neutral usage-publisher create comment (69).

Note (Pariston, Ryosuke, CRF-3). The usage_event create hunk in dbauthz.go is identical to the one in #27312, so the two land in either order without conflict as the description states. Flagging only so a future reviewer does not read the duplication as a rebase artifact, and so a later divergence between the two copies is not silent.

Note (Netero, CRF-4). cron_test.go:67 carries a pre-existing em-dash (U+2014) in a comment. It is out of this diff and out of CI's diff-scoped em-dash check, but since the author is already editing this file, fixing it here is cheap.

🤖 This review was automatically generated with Coder Agents.

Comment thread enterprise/cli/server.go
err = usageCron.Register(usage.CronJob{
Name: "ai-seats",
Interval: usage.AISeatsInterval,
EventType: usagetypes.UsageEventTypeHBAISeatsV1,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [CRF-1] CronJob.EventType duplicates the type Fn already produces, and the two are reconciled only at runtime. (Meruem, Ryosuke)

Register validates job.EventType.IsHeartbeat() but cannot check that EventType matches what Fn returns, because Fn hits the database and cannot run at registration time. The match is instead verified inside cron.run via event.EventType() != job.EventType, where a mismatch logs an error and continues. That is the same failure class this PR fixes (a job that is configured but emits zero usage events), relocated from registration to execution: a future job whose EventType disagrees with its Fn would silently stop billing events with only a warning line.

Correct today: ai-seats is the only registered job, EventType matches AISeatsHeartbeat's output, and TestAISeatsHeartbeat pins that output. No change requested for this PR. If a second job is ever added, a table-driven test asserting each registered job's EventType equals its Fn's produced type would make the invariant mechanical rather than trusting the runtime log.

🤖

jaaydenh added a commit that referenced this pull request Jul 30, 2026
…time (#27312)

closes CODAGT-839
closes CODAGT-843
closes CODAGT-773

## Summary

Adds a new heartbeat usage event type, `hb_agent_runtime_v1`, measuring
the total agent-loop runtime of Coder Agents (chats) per UTC hour, plus
a reconciler that generates one event per hour with self-healing
backfill over a trailing 7-day window. Events flow to Tallyman through
the existing publisher unchanged. This measures the new Coder Agents
(the `chats` tables), not the deprecated Tasks counted by
`dc_managed_agents_v1`.

Independent of #27508, which fixes the dead ai-seats cron registration.
Both PRs carry the identical `usage_event` create permission hunk for
the usage-publisher subject (this feature's generator and the ai-seats
cron each need it for heartbeat inserts), so they can land in either
order and the overlap merges cleanly.

> [!WARNING]
> **Do not include this in a release until Tallyman accepts
`hb_agent_runtime_v1`.** The publisher marks permanently rejected events
as done-forever, and the generator then sees those buckets as complete
locally, so their usage would be silently and permanently lost.

## Details

Each event's payload is `{"runtime_ms": N}`: the sum of
`chat_messages.runtime_ms` for messages created in the hour bucket `[H,
H+1)`, across all chats (sub-agents, API-created, archived, and
soft-deleted messages included). Events use deterministic IDs
(`hb_agent_runtime_v1:<bucket start>`) with `created_at` set to the
bucket start, so concurrent replicas race safely via `ON CONFLICT (id)
DO NOTHING` without locking, and daily rollups attribute backfilled
hours to the correct day. Idle hours produce zero-valued events. A
bucket becomes eligible 5 minutes after it closes; hours missing for
longer than the 7-day window are forfeited, which can only undercount.

Note that this makes `usage_events.created_at` explicitly the *event
occurrence time* rather than the row insertion time; the two only
diverge for backfilled events. It already behaved as the occurrence
timestamp (it drives the daily rollup day and is shipped to
Tallyman/Metronome as the event timestamp), and the migration now
documents this with a `COMMENT ON COLUMN`, which also surfaces as a Go
doc comment on `UsageEvent.CreatedAt`.

The new `usage.Generator` runs unconditionally in enterprise builds; the
`publish_usage_data` license flag continues to gate egress only, so
air-gapped deployments still fill their local ledger. The
`aggregate_usage_event()` trigger sums `runtime_ms` per day into
`usage_events_daily` (unlike `hb_ai_seats_v1`, which takes the daily
max).

`InsertHeartbeatUsageEvent` now takes an explicit `createdAt` so
generators can backfill historical buckets; the cron passes
`clock.Now()` to preserve its existing behavior.

## Tallyman follow-up

<details>
<summary>Prompt for the Tallyman-repo agent</summary>

> **Task**: Add support for the new Coder usage event type
`hb_agent_runtime_v1` so Tallyman accepts, validates, and forwards it to
Metronome.
>
> **Background**: coder/coder PR (this PR) adds hourly heartbeat events
measuring Coder Agent runtime. Events arrive via the existing
`/api/v1/events/ingest` endpoint with: `event_type:
"hb_agent_runtime_v1"`, `event_data: {"runtime_ms": <int64 >= 0>}`,
deterministic `id` of the form `hb_agent_runtime_v1:2026-07-15_14:00:00`
(UTC hour bucket start), and `created_at` set to the bucket start (may
be up to ~8 days in the past due to backfill; within Metronome's 34-day
dedup window). Zero-value events are normal (idle hours).
>
> **Work**:
> 1. Update Tallyman's vendored/imported `coderd/usage/usagetypes` (or
equivalent) to the coder/coder commit that adds
`UsageEventTypeHBAgentRuntimeV1` and `HBAgentRuntime`.
> 2. Ensure ingestion validation accepts the type (`Valid()` switches)
and rejects negative `runtime_ms`.
> 3. Ensure Metronome forwarding maps the event with transaction ID
derived from the event `id` as for existing types, passing `runtime_ms`
through as the property for a SUM-aggregated billable metric ("Coder
Agent Hours" = `SUM(runtime_ms) / 3,600,000`).
> 4. Do NOT permanently reject unknown-but-well-formed future `hb_*`
types if avoidable; at minimum confirm current behavior for unknown
types (temporary vs permanent rejection) and report it.
> 5. Tests: ingest accept/validate, dedup by ID, Metronome payload
mapping.
>
> **Constraint**: this must be deployed to tallyman-prod **before** any
coder/coder release containing the event generator; coderd treats
permanent rejections as terminal per event.

</details>
@github-actions github-actions Bot added the stale This issue is like stale bread. label Aug 18, 2026
@github-actions github-actions Bot closed this Aug 21, 2026
@jaaydenh jaaydenh reopened this Aug 22, 2026
@github-actions github-actions Bot removed the stale This issue is like stale bread. label Aug 23, 2026
@github-actions github-actions Bot added the stale This issue is like stale bread. label Aug 30, 2026
@github-actions github-actions Bot closed this Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant