feat: fall back to the Everyone group for AI spend attribution - #27364
Conversation
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
| organizations.is_default DESC, -- prefer the default org | ||
| organizations.name ASC -- organization name tiebreak |
There was a problem hiding this comment.
The logic for the Everyone fallback is as follows:
- If the user belongs to the default organization, use the
Everyonegroup from that org. - If the user does not belong to the default organization and belongs to multiple orgs, use the org that comes first alphabetically (by name).
GetHighestGroupAIBudgetByUser was also updated to break ties by org name and then group name, for the case where a user belongs to two orgs that each have a same-named group with the same budget. This keeps tie-breaking consistent across the codebase.
This logic was chosen because ordering by names is easier for customers to reason about than ordering by ids. The trade-off is that org and group names are mutable, so the resolved effective group can change on rename. For example, a user in org-a and org-b resolves to org-a's Everyone group; if org-a is later renamed to org-c, the effective group shifts to org-b's Everyone group.
This is an edge case, but we should aim for behavior that is less surprising and consistent across the codebase. Names are mutable, so renames shift the result (however, this also depends on how frequent this sort of operation is 🤔 ). As a result, maybe the initial approach of ordering by an immutable key such as the first org joined (organization_members.created_at) would be a better approach. This would be stable under renames and new memberships, changing only if the user leaves that org. wdyt?
There was a problem hiding this comment.
If the user belongs to both the default organization and multiple orgs, should we behave as if they were not a member of the default organization?
I'm skeptical on using name ordering as a tie-breaker. organization_members.created_at seems like a more stable solution.
There was a problem hiding this comment.
If the user belongs to both the default organization and multiple orgs, should we behave as if they were not a member of the default organization?
No, whenever the user belongs to the default organization, that will be the one used (even in a multi-org scenario). The step 2, is only when the user does not belong to the default organization and belongs to multiple orgs.
I'm skeptical on using name ordering as a tie-breaker. organization_members.created_at seems like a more stable solution.
ok, so I will change this 👍 Thanks
There was a problem hiding this comment.
Updated in d42be83
We still need to order by organizations.id as a final tiebreaker, for the (rare) case a member has organization_members.created_at timestamp for 2 different orgs. GetHighestGroupAIBudgetByUser was updated as well with the same logic.
| if errors.Is(err, sql.ErrNoRows) { | ||
| // This should not happen, as a user should always be a member of an | ||
| // organization and its associated Everyone group. | ||
| return EffectiveGroup{}, false, nil |
There was a problem hiding this comment.
AFAIK, a user should always belong to an org. However, it seems the codebase allows a user to end up orgless in a few cases, e.g. an admin removing a user from their last org (only self-removal is blocked), or the user's only org being deleted. This shouldn't happen in normal circumstances, but the code handles it defensively nevertheless.
There was a problem hiding this comment.
Good call 👍
For the record, it looks like in most cases a 'dis-organized' user can't really do anything of much substance unless they have a site-wide admin role.
|
/coder-agents-review |
|
Chat: Review posted | View chat Review history
deep-review v0.9.0 | Round 1 | Last posted: Round 1, 7 findings (3 P3, 1 P4, 1 Nit, 2 Note), COMMENT. Review Finding inventoryFinding inventory - PR #27364Findings
Contested and acknowledgedNone this round. Round logRound 1Netero first pass (1 Note, CRF-1). Mechanical floor clean, panel spawned (16 reviewers: bisky, hisoka, mafu-san, mafuuu, pariston, ging-go, gon, leorio, knuckle, kurapika, komugi, chopper, ryosuke, killua, knov, + wildcards meruem, razor). Multi-domain (Go + SQL + auth/billing). No P0-P2. Centerpiece: CRF-2, a 7-reviewer P3 convergence on lost ORDER BY determinism in the highest-group query, verified against dump.sql:4839 (partial unique index) and the query text. CRF-3 daily-spend skip verified at aibridgedserver.go:382. ging-go and kurapika found no code defects (kurapika confirmed auth gate + tenant boundary correct). Reviewed against 9bd4cf2..a932e25. Event COMMENT. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
This is a clean, well-scoped change. Splitting resolution into ResolveUserAIBudget (enforcement, no fallback) and ResolveUserEffectiveGroup (attribution, with the unlimited Everyone fallback) keeps the blocking path untouched, and reusing the Everyone group (group_id == organization_id) attributes to the org without a schema change. The auth gate on the new query matches its siblings exactly, the tenant boundary in GetGroupMembersAISpend still holds, and test density is high with real DB fixtures that would actually flip if the ordering were wrong. Pariston tried to break the premise and couldn't; kurapika and ging-go found no defects.
Severity counts: 0 P0-P2, 3 P3, 1 P4, 1 Nit, 3 Note.
The one finding worth real attention (CRF-2) is a seven-reviewer convergence: GetHighestGroupAIBudgetByUser dropped its budget.group_id ASC final tiebreak and now leans on organizations.name for determinism, but org names are unique only among non-deleted orgs (idx_organization_name_lower is UNIQUE (lower(name)) WHERE deleted = false) and the query never filters deleted orgs. The guarding test TieBrokenByGroupID was removed in this PR. The fix is nearly free (restore group_id ASC and/or add AND organizations.deleted = false in both the query and the user_highest_group CTE) and restores an invariant the old code held on purpose. This affects enforcement, not just attribution, which is why it leads.
Two description-accuracy notes, not code changes: the description says spend is "now always attributed and tracked," but a user with zero org membership still records spend with a NULL group (logged as unexpected, handled honestly in code) - consider "always attributed when the user has org membership." And CRF-8: enabling an Everyone-group budget mid-window now retroactively counts spend recorded earlier in the window, which "enforcement is unaffected" does not mention.
Hisoka summed the panel's mood: "I came to fight this one. It fought back well... I looked for the kill and it wasn't here. The design held."
CRF-4 (the unreachable Limit == nil guard) drew eight reviewers as a Note: harmless today, but it defends a state ResolveUserAIBudget cannot return, and if a future caller swaps in ResolveUserEffectiveGroup here it would silently disable enforcement for every unbudgeted user. A one-line comment naming the invariant, or dropping the dead operand, would make the boundary explicit.
🤖 This review was automatically generated with Coder Agents.
f539d3a to
dfb8399
Compare
There was a problem hiding this comment.
suggestion, non-blocking: suggest re-using a single dbtestutil.NewDB instance where possible across tests. Potential follow-up.
There was a problem hiding this comment.
Will address this in a follow-up PR
1f110b2 to
d42be83
Compare
| func (q *querier) GetUserEveryoneFallbackGroup(ctx context.Context, userID uuid.UUID) (uuid.UUID, error) { | ||
| if _, err := q.GetUserByID(ctx, userID); err != nil { // AuthZ check | ||
| return uuid.Nil, err | ||
| } |
There was a problem hiding this comment.
non-blocking: Would a more targeted authz check be better here? This reads a bunch of fields from the users table when it's not specifically required. Does this end up being called with the user's actor in context, or do we use a special aibridged actor? Again, this could be a potential follow-up but might reduce database load.
There was a problem hiding this comment.
This is a good point. This actually follows the same pattern as the other 2 queries that we use on user budget resolution: GetUserAIBudgetOverride and GetHighestGroupAIBudgetByUser. So we might need to update all of them.
This is used in 2 paths:
- interception cost recording: this is done as the aibridged daemon actor
- endpoint
GET /users/{user}/ai/spend: this is done as the requesting user's actor
We might be able to do something like:
if err := q.authorizeContext(ctx, policy.ActionRead, rbac.ResourceUser.WithID(userID).WithOwner(userID.String())); err != nil {
return uuid.Nil, err
}
The one difference is that we no longer confirm the user exists, so we can't distinguish "user doesn't exist" from "user has no org membership". I think that's fine: it only matters on path 2, and IIRC that endpoint already resolves the user through a middleware before this runs. I'll address it in a follow-up. Thanks for raising it!
Description
Previously, a user with no per-user override and no membership in a budgeted group had no effective group, so their AI spend was attributed nowhere and was, therefore, untracked. This change falls back to the organization's Everyone group when no override or group budget applies.
Since every user in an organization is implicitly a member of that org's Everyone group, spend is now attributed and tracked for any user with organization membership. A user with no organization membership resolves to no group, so their daily spend is not incremented and a warning is logged.
The fallback is unlimited, so enforcement is unaffected: only override and group budgets can block requests. For users in multiple organizations, an existing budget on any Everyone group is still chosen by the "highest" policy; when none is budgeted, the fallback prefers the default org, then orders by earliest organization membership.
Changes
ResolveUserEffectiveGroupand theGetUserEveryoneFallbackGroupquery: resolve override → group budget → Everyone group fallback.GetGroupMembersAISpendto surface the Everyone fallback as the effective group.GetHighestGroupAIBudgetByUserto break ties by earliest organization membership, then group ID, keeping multi-org resolution deterministic and consistent with the fallback.Closes https://linear.app/codercom/issue/AIGOV-509/fall-back-to-the-everyone-group-for-spend-attribution
Note
Initially generated by Claude Opus 4.7, modified and reviewed by @ssncferreira