Thanks to visit codestin.com
Credit goes to github.com

Skip to content

perf(coderd/rbac): build span role attributes only when recording - #27310

Merged
jeremyruppel merged 1 commit into
jeremy/devex-608-authcheck-batchfrom
jeremy/devex-608-rbac-span-attrs
Aug 6, 2026
Merged

perf(coderd/rbac): build span role attributes only when recording#27310
jeremyruppel merged 1 commit into
jeremy/devex-608-authcheck-batchfrom
jeremy/devex-608-rbac-span-attrs

Conversation

@jeremyruppel

@jeremyruppel jeremyruppel commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

rbacTraceAttributes materialized the subject's role names (one string allocation per role) and was passed into every Filter, Authorize, and Prepare span at creation time, so the O(roles) work ran even when no tracer was recording. It also called SafeRoleNames() twice.

Replace it with setRBACAttributes, which attaches the same attributes after the span is created and only when span.IsRecording() is true, reading SafeRoleNames() once. Recorded spans are unchanged; untraced and unsampled calls skip the per-role work.

This originated from #27309: once /authcheck checks are batched through rbac.Filter, each below-threshold group paid the role-attribute build for the Filter span and for every per-object Authorize span, so the redundant per-call work showed up as extra allocations per request.

Benchmarks

AMD EPYC 9575F, benchstat, no tracer configured (exercises the IsRecording()==false path).

BenchmarkRBACManyOrgs (general RBAC eval), before vs after: wall time flat (geomean −0.04%), allocations strictly lower everywhere (geomean B/op −0.52%; Authorize −1.0 to −1.3% B/op), no regressions.

Authcheck path (BenchmarkAuthcheckGrouping, #27309 vs #27310, back-to-back): this change is an allocation reduction and is time-neutral. On the endpoint (Grouped) path, per-request allocations drop ~4-5% B/op at common org counts (1-10); on the pure per-object path the reduction grows with org count (B/op −2.6% → −7.3% at 100 orgs). Wall time is flat within noise: low-org deltas sit inside this host's ±10-23% run-to-run variance, so no wall-time claim is made.

Net: same speed, less garbage per request, which also lowers GC pressure under real concurrent load.

Decision log
  • The Filter span wraps the whole filtering routine (total latency + num_objects); it is the valuable span and is kept. The costly part was rbacTraceAttributes, not the span itself.
  • rbacTraceAttributes was O(roles): it allocated a string per role for the subject_roles attribute and called SafeRoleNames() twice. On Filter's below-threshold fallback it ran once for the Filter span and again for each per-object Authorize span, so a group of N objects paid N+1 builds vs the old loop's N. Benchmarks confirm this as real per-call allocation; its wall-time cost is below the authcheck benchmark's noise floor.
  • Deferring attribute construction behind IsRecording() requires the span object, so the three callsites moved from StartSpan(ctx, rbacTraceAttributes(...)) to StartSpan(ctx) then setRBACAttributes(span, ...). No spans were removed or renamed; recorded output is identical.
  • Tradeoff: when a span is not recording, subject_roles/num_subject_roles/etc. are not computed. Unsampled spans emit nothing anyway, so there is no observable output change.

Authored with Coder Agents.

@linear-code

linear-code Bot commented Jul 16, 2026

Copy link
Copy Markdown

DEVEX-608

jeremyruppel commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

Copy link
Copy Markdown
Contributor Author

Authcheck-path benchmark (#27309 vs #27310)

BenchmarkAuthcheckGrouping models the full /authcheck check set (the ~15 (action, resource type) pairs per org from organizations.ts, flattened across N orgs, grouped and filtered). Run back-to-back on the same host, -benchmem -benchtime 200ms -count 10. Grouped is the path the endpoint actually runs.

Effect of this PR on the endpoint (Grouped) path:

orgs sec/op B/op allocs/op
1 −1.8% −4.7% −2.0%
5 ~ (noise) −4.6% −1.7%
10 ~ (noise) −5.2% −1.7%
50 ~ −0.5% −0.15%
100 +2.3% −0.6% −0.16%

This change is an allocation reduction and is time-neutral. Wall time is flat within noise: the small low-org deltas and the +2.3% blip at 100 orgs (whose allocations are flat) sit inside this shared host's run-to-run variance, which swings ±10-23% at low org counts. The reproducible, clean signal (allocation counts have ±0% variance) is fewer bytes and allocations per request: ~4-5% less on the endpoint path at common org counts (1-10). On the pure per-object path the reduction grows with org count (B/op −2.6% → −7.3% at 100 orgs), since it removes the per-Authorize role-name slice build.

This confirms the mechanism: the removed rbacTraceAttributes work was real per-call allocation, previously paid even without a tracer recording. Its wall-time cost is below this benchmark's noise floor, so the honest claim is "same speed, less garbage," which also lowers GC pressure under real concurrent load.

Full benchstat (#27309 vs #27310)
goos: linux
goarch: amd64
pkg: github.com/coder/coder/v2/coderd
cpu: AMD EPYC 9575F 64-Core Processor               
                                         │   pr27309    │              pr27310               │
                                         │    sec/op    │   sec/op     vs base               │
AuthcheckGrouping/PerObject/orgs=1-128     891.9µ ± 12%   867.7µ ± 4%  -2.72% (p=0.019 n=10)
AuthcheckGrouping/Grouped/orgs=1-128       903.8µ ±  1%   887.4µ ± 3%  -1.81% (p=0.043 n=10)
AuthcheckGrouping/PerObject/orgs=5-128     6.850m ±  3%   6.862m ± 2%       ~ (p=0.796 n=10)
AuthcheckGrouping/Grouped/orgs=5-128       6.875m ±  1%   6.998m ± 2%       ~ (p=0.123 n=10)
AuthcheckGrouping/PerObject/orgs=10-128    21.04m ±  4%   20.54m ± 2%  -2.36% (p=0.015 n=10)
AuthcheckGrouping/Grouped/orgs=10-128      20.77m ±  1%   20.76m ± 5%       ~ (p=0.971 n=10)
AuthcheckGrouping/PerObject/orgs=50-128    494.0m ±  1%   502.9m ± 2%  +1.80% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=50-128      357.9m ± 13%   344.8m ± 1%       ~ (p=0.052 n=10)
AuthcheckGrouping/PerObject/orgs=100-128    2.598 ±  7%    2.634 ± 1%       ~ (p=0.105 n=10)
AuthcheckGrouping/Grouped/orgs=100-128     951.6m ±  1%   973.5m ± 1%  +2.30% (p=0.000 n=10)
geomean                                    38.55m         38.42m       -0.34%

                                         │   pr27309    │               pr27310               │
                                         │     B/op     │     B/op      vs base               │
AuthcheckGrouping/PerObject/orgs=1-128     498.8Ki ± 0%   486.1Ki ± 0%  -2.56% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=1-128       521.3Ki ± 0%   496.6Ki ± 0%  -4.74% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=5-128     3.275Mi ± 0%   3.147Mi ± 0%  -3.90% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=5-128       3.328Mi ± 0%   3.176Mi ± 0%  -4.55% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=10-128    8.748Mi ± 0%   8.333Mi ± 0%  -4.74% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=10-128      8.838Mi ± 0%   8.382Mi ± 0%  -5.15% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=50-128    130.1Mi ± 0%   121.2Mi ± 0%  -6.83% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=50-128      64.79Mi ± 0%   64.44Mi ± 0%  -0.54% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=100-128   476.4Mi ± 0%   441.8Mi ± 0%  -7.28% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=100-128     115.4Mi ± 0%   114.7Mi ± 0%  -0.62% (p=0.000 n=10)
geomean                                    12.55Mi        12.03Mi       -4.12%

                                         │   pr27309   │              pr27310               │
                                         │  allocs/op  │  allocs/op   vs base               │
AuthcheckGrouping/PerObject/orgs=1-128     12.54k ± 0%   12.42k ± 0%  -0.96% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=1-128       12.78k ± 0%   12.53k ± 0%  -2.00% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=5-128     92.38k ± 0%   91.10k ± 0%  -1.38% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=5-128       92.81k ± 0%   91.27k ± 0%  -1.67% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=10-128    258.8k ± 0%   254.8k ± 0%  -1.56% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=10-128      259.5k ± 0%   255.0k ± 0%  -1.72% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=50-128    4.215M ± 0%   4.134M ± 0%  -1.90% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=50-128      2.232M ± 0%   2.229M ± 0%  -0.15% (p=0.000 n=10)
AuthcheckGrouping/PerObject/orgs=100-128   15.72M ± 0%   15.40M ± 0%  -1.98% (p=0.000 n=10)
AuthcheckGrouping/Grouped/orgs=100-128     4.044M ± 0%   4.037M ± 0%  -0.16% (p=0.000 n=10)
geomean                                    375.1k        370.1k       -1.35%

Authored with Coder Agents.

@Emyrk Emyrk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍

@johnstcn
johnstcn removed their request for review July 20, 2026 13:32
@jeremyruppel
jeremyruppel force-pushed the jeremy/devex-608-rbac-span-attrs branch from 92a7289 to 8ca4355 Compare July 27, 2026 13:15
@jeremyruppel
jeremyruppel force-pushed the jeremy/devex-608-rbac-span-attrs branch 3 times, most recently from 6bb1809 to 2c9356d Compare July 27, 2026 18:32
rbacTraceAttributes materialized the subject's role names (one string
allocation per role) and was passed into every Filter, Authorize, and
Prepare span at creation time, so the O(roles) work ran even when no
tracer was recording. It also called SafeRoleNames twice.

Replace it with setRBACAttributes, which attaches the same attributes
after the span is created and only when span.IsRecording() is true, and
reads SafeRoleNames once. Recorded spans are unchanged; untraced and
unsampled calls skip the per-role work. This removes the per-group
overhead that made small /authcheck requests slower once checks were
batched through Filter.
@jeremyruppel
jeremyruppel force-pushed the jeremy/devex-608-rbac-span-attrs branch from 2c9356d to 440e7ae Compare July 27, 2026 19:22
@github-actions github-actions Bot added the stale This issue is like stale bread. label Aug 4, 2026
@jeremyruppel
jeremyruppel merged commit d9d6ce9 into main Aug 6, 2026
33 of 55 checks passed
@jeremyruppel
jeremyruppel deleted the jeremy/devex-608-rbac-span-attrs branch August 6, 2026 13:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants