Commit 7e708b2
authored
perf(coderd/rbac): collapse org authorization to a set-membership test (#27244)
## Problem
Authorization for users who belong to many organizations is slow. On the
list <br>endpoints (`/api/v2/organizations`, `/users`, `/groups`) a user
in hundreds of <br>orgs saw multi-second page loads
<br>([DEVEX-608](https://linear.app/codercom/issue/DEVEX-608/performance-degrades-for-users-in-many-organizations-across-multiple)
<br>/ #21890 / Pylon
[#2758](<#2758>)). This is
partial-evaluation bound: `rbac.Prepare` <br>scales with the number of
org-scoped roles the subject carries.
## Root cause
The known-org path in `check_org_permissions` indexed an N-entry vote
map by the <br>object's org id:
```rego
vote := allow_map[input.object.org_owner]
```
`input.object.org_owner` is unknown during partial evaluation. Indexing
a map by <br>an unknown key cannot reduce to a single expression, so OPA
emits one residual <br>query per org membership, and
`newPartialAuthorizer` then calls `PrepareForEval` <br>once per
residual, making `Prepare` O(N) in org count. The list endpoints
<br>intentionally use partial eval; the fan-out is in partial eval
itself.
## Change
Test the object's org id for membership in a set that is fully known at
<br>partial-evaluation time, so the query collapses to a single
<br>`organization_id = ANY(ARRAY[...])` residual instead of N residuals:
* The known-org clause only ever votes to allow, tested via
<br>`org_owner in org_ids_with_vote(role_org_votes, 1)`.
* Org-level denies are folded into the org-member level as a ground set
<br>difference (`member_allow - org_deny`), so the unknown org id
appears in only <br>one positive membership test and the decision never
branches on it.
* The per-org vote maps are computed once as memoized zero-arg rules
<br>(`role_org_votes`, `role_member_votes`, `scope_org_votes`,
<br>`scope_member_votes`) instead of through parametrized functions that
OPA <br>re-evaluates at every call site.
* `role_allow`/`scope_allow`, the `any_org` path, and full evaluation
are <br>unchanged in behavior.
Semantics are unchanged (see the equivalence argument below). The only
<br>representational change is that a denied known org's intermediate
`org` vote is <br>now `0` instead of `-1`, compensated by the set
difference and not observable in <br>the final `allow` decision.
## Results
Measured with `BenchmarkRBACManyOrgs` (added on `main` in
#27270). Full tables: <br>[B/op and
allocs/op](<#27244 (comment)>).
* Residual queries: O(N) -> O(1).
* `Prepare` / `PrepareAndCompile` memory changes from < />quadratic
growth on `main` <br>(176 MiB, 7.08M allocs per op at 100 orgs) to
near-linear (6.5 MiB, 258k <br>allocs), a < />96% reduction at 100 orgs,
with similar wins in time.
* Memoizing the vote maps removed an early single-org regression: at 1
org <br>`Prepare` now allocates < />7% fewer bytes and < />9% fewer
objects than `main`.
* `Authorize` (full evaluation) memory is marginally higher (+1-8%,
largest at <br>1 org) and time-neutral. This is the inherent cost of the
set-membership form <br>that keeps partial evaluation from fanning out;
full evaluation builds an <br>allow set it would not otherwise need.
* `go test ./coderd/rbac/...` passes, including `TestAuthorizeDomain`
(full- vs <br>partial-eval equivalence) and the regosql suite.
A second, independent bottleneck remains (out of scope here): the vote
map is <br>still built in O(N^2) in `check_all_org_permissions`
<br>(`roles[_].by_org_id[org_id]` scans all roles per org). Fixing it
means <br>pre-merging roles' `by_org_id` into one org->perms map in the
OPA input, and is <br>tracked as a follow-up.
## Testing
* `OrgDenyBlocksMember` (`TestAuthorizeLevels`): an org-level deny
blocks a <br>member-allowed action on an owned in-org object, while a
clean org is allowed, <br>including an action-scoped deny.
* `ScopeOrgDenyBlocksMember` (`TestAuthorizeScope`): the same fold at
the scope <br>level.
* The shared harness covers full and partial evaluation and asserts the
partial <br>result compiles to SQL with zero support rules.
<details><summary>Decision log and equivalence argument</summary>
### Why not deny-via-set-membership
The first attempt expressed deny as a second set-membership clause (`:=
-1 if org_owner in deny_set`). That makes `org`/`scope_org`
multi-valued, and the `not org = -1` checks in
`role_allow`/`scope_allow` then cause OPA to emit a
`data.partial.__not__` support rule that regosql cannot compile
(`TestAuthorizeDomain/UserACLList` failed). It failed even when the deny
set was empty, purely because the `-1` clause exists.
### Why not deny-via-enumeration
A follow-up enumerated only the (usually empty) deny set. It compiled
and passed, but it branches on the unknown org id (one ground residual
per denied org), which violates the "do not branch on the unknown" rule
in `coderd/rbac/POLICY.md`.
### Final approach: allow-only + ground set difference
The known-org clause votes only to allow, and the org-level deny gate is
moved into the org-member level as `member_allow - org_deny`, a set
difference over fully-known sets. The unknown org id is used only in
positive `in` tests, so there is no enumeration, no negated membership,
and no branching on the unknown.
### Empty-set residual pruning
A naive set-membership left unsatisfiable residuals (`org_owner in
set()`) for levels with no matching permissions (e.g. the org level for
an org-member role, or scope-org for `ScopeAll`), each still costing a
`PrepareForEval`. Guarding each membership with a ground `count(...) >
0` lets OPA drop those branches, flattening the residual count across
org sizes.
### Memoized vote maps
Profiling the single-org path showed the cost was repeated function
evaluation: the parametrized helpers rebuilt the same vote map for the
org, member, and scope paths on every check. Hoisting the maps into
memoized zero-arg complete rules (which OPA evaluates once per query)
removed that overhead and eliminated the single-org `Prepare`
regression, while composition keeps the policy readable.
### Equivalence (known-org path, `site != -1`)
* A: original `org == 1` <=> `org_owner in org_allow` (unchanged).
* B: original `org != -1 and member == 1` <=> `org_owner not in org_deny
and org_owner in member_allow` <=> `org_owner in (member_allow -
org_deny)` = new `org_member == 1`.
The critical case (`org` denies, member allows): old blocks it via `not
org = -1`; new blocks it because `org_owner` is removed from
`member_allow - org_deny`. Same outcome. Deny-wins aggregation is intact
because `check_all_org_permissions` still nets an org to `-1` via
`to_vote`, landing it in `org_deny`.
</details>
---
This PR was generated by Coder Agents on behalf of @jeremyruppel.1 parent f9047e5 commit 7e708b2
3 files changed
Lines changed: 268 additions & 41 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
68 | 68 | | |
69 | 69 | | |
70 | 70 | | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
71 | 95 | | |
72 | 96 | | |
73 | 97 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1009 | 1009 | | |
1010 | 1010 | | |
1011 | 1011 | | |
| 1012 | + | |
| 1013 | + | |
| 1014 | + | |
| 1015 | + | |
| 1016 | + | |
| 1017 | + | |
| 1018 | + | |
| 1019 | + | |
| 1020 | + | |
| 1021 | + | |
| 1022 | + | |
| 1023 | + | |
| 1024 | + | |
| 1025 | + | |
| 1026 | + | |
| 1027 | + | |
| 1028 | + | |
| 1029 | + | |
| 1030 | + | |
| 1031 | + | |
| 1032 | + | |
| 1033 | + | |
| 1034 | + | |
| 1035 | + | |
| 1036 | + | |
| 1037 | + | |
| 1038 | + | |
| 1039 | + | |
| 1040 | + | |
| 1041 | + | |
| 1042 | + | |
| 1043 | + | |
| 1044 | + | |
| 1045 | + | |
| 1046 | + | |
| 1047 | + | |
| 1048 | + | |
| 1049 | + | |
| 1050 | + | |
| 1051 | + | |
| 1052 | + | |
| 1053 | + | |
| 1054 | + | |
| 1055 | + | |
| 1056 | + | |
| 1057 | + | |
| 1058 | + | |
| 1059 | + | |
| 1060 | + | |
| 1061 | + | |
| 1062 | + | |
| 1063 | + | |
| 1064 | + | |
| 1065 | + | |
| 1066 | + | |
| 1067 | + | |
| 1068 | + | |
| 1069 | + | |
| 1070 | + | |
| 1071 | + | |
| 1072 | + | |
| 1073 | + | |
| 1074 | + | |
| 1075 | + | |
| 1076 | + | |
| 1077 | + | |
| 1078 | + | |
| 1079 | + | |
| 1080 | + | |
| 1081 | + | |
| 1082 | + | |
| 1083 | + | |
| 1084 | + | |
| 1085 | + | |
| 1086 | + | |
| 1087 | + | |
| 1088 | + | |
| 1089 | + | |
| 1090 | + | |
| 1091 | + | |
| 1092 | + | |
1012 | 1093 | | |
1013 | 1094 | | |
1014 | 1095 | | |
| |||
1341 | 1422 | | |
1342 | 1423 | | |
1343 | 1424 | | |
| 1425 | + | |
| 1426 | + | |
| 1427 | + | |
| 1428 | + | |
| 1429 | + | |
| 1430 | + | |
| 1431 | + | |
| 1432 | + | |
| 1433 | + | |
| 1434 | + | |
| 1435 | + | |
| 1436 | + | |
| 1437 | + | |
| 1438 | + | |
| 1439 | + | |
| 1440 | + | |
| 1441 | + | |
| 1442 | + | |
| 1443 | + | |
| 1444 | + | |
| 1445 | + | |
| 1446 | + | |
| 1447 | + | |
| 1448 | + | |
| 1449 | + | |
| 1450 | + | |
| 1451 | + | |
| 1452 | + | |
| 1453 | + | |
| 1454 | + | |
| 1455 | + | |
| 1456 | + | |
| 1457 | + | |
| 1458 | + | |
| 1459 | + | |
| 1460 | + | |
| 1461 | + | |
| 1462 | + | |
| 1463 | + | |
| 1464 | + | |
| 1465 | + | |
| 1466 | + | |
| 1467 | + | |
| 1468 | + | |
| 1469 | + | |
| 1470 | + | |
| 1471 | + | |
| 1472 | + | |
| 1473 | + | |
| 1474 | + | |
| 1475 | + | |
| 1476 | + | |
| 1477 | + | |
| 1478 | + | |
| 1479 | + | |
1344 | 1480 | | |
1345 | 1481 | | |
1346 | 1482 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
99 | 99 | | |
100 | 100 | | |
101 | 101 | | |
102 | | - | |
103 | | - | |
104 | | - | |
105 | | - | |
106 | | - | |
107 | | - | |
108 | | - | |
109 | | - | |
110 | 102 | | |
111 | 103 | | |
112 | | - | |
113 | | - | |
114 | | - | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
115 | 109 | | |
116 | 110 | | |
117 | 111 | | |
118 | | - | |
119 | | - | |
120 | | - | |
121 | | - | |
122 | | - | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
123 | 115 | | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
124 | 119 | | |
125 | 120 | | |
126 | 121 | | |
| |||
137 | 132 | | |
138 | 133 | | |
139 | 134 | | |
140 | | - | |
141 | | - | |
142 | | - | |
143 | | - | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
144 | 143 | | |
145 | | - | |
| 144 | + | |
146 | 145 | | |
147 | | - | |
148 | | - | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
| 160 | + | |
| 161 | + | |
| 162 | + | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
149 | 170 | | |
150 | 171 | | |
151 | | - | |
152 | | - | |
153 | | - | |
154 | | - | |
155 | | - | |
156 | | - | |
157 | | - | |
158 | | - | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
159 | 175 | | |
| 176 | + | |
| 177 | + | |
160 | 178 | | |
161 | | - | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
162 | 187 | | |
163 | | - | |
164 | | - | |
165 | | - | |
166 | | - | |
167 | | - | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
168 | 191 | | |
169 | 192 | | |
170 | 193 | | |
| |||
190 | 213 | | |
191 | 214 | | |
192 | 215 | | |
193 | | - | |
194 | | - | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
195 | 219 | | |
196 | 220 | | |
197 | 221 | | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
198 | 244 | | |
199 | 245 | | |
200 | 246 | | |
201 | 247 | | |
202 | | - | |
| 248 | + | |
| 249 | + | |
203 | 250 | | |
204 | 251 | | |
205 | 252 | | |
206 | 253 | | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
| 258 | + | |
| 259 | + | |
| 260 | + | |
| 261 | + | |
| 262 | + | |
| 263 | + | |
| 264 | + | |
| 265 | + | |
207 | 266 | | |
208 | 267 | | |
209 | 268 | | |
210 | 269 | | |
211 | | - | |
| 270 | + | |
| 271 | + | |
212 | 272 | | |
213 | 273 | | |
214 | 274 | | |
| |||
243 | 303 | | |
244 | 304 | | |
245 | 305 | | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
246 | 310 | | |
247 | 311 | | |
248 | 312 | | |
| |||
290 | 354 | | |
291 | 355 | | |
292 | 356 | | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
293 | 360 | | |
294 | 361 | | |
295 | 362 | | |
| |||
0 commit comments