Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat: resolve bedrock inference profiles when a provider is written - #29112

Merged
evgeniy-scherbina merged 26 commits into
mainfrom
yevhenii/aigov-488-resolve-inference-profiles-at-write-time
Sep 11, 2026
Merged

feat: resolve bedrock inference profiles when a provider is written#29112
evgeniy-scherbina merged 26 commits into
mainfrom
yevhenii/aigov-488-resolve-inference-profiles-at-write-time

Conversation

@evgeniy-scherbina

@evgeniy-scherbina evgeniy-scherbina commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #28877, which resolved Bedrock application inference profile (AIP) ARNs while constructing a provider in the gateway. Review found two consequences of putting an AWS call on that path: a transient failure dropped the provider from the reloaded snapshot until the next provider change or restart, and the initial reload blocked coder server startup for up to 30s per AIP-configured provider under an uncancelable context.

Resolution now happens once, in coderd, when a provider is created or updated. The result is stored with the provider and travels to the gateway in the existing provider payload, so the gateway makes no Bedrock control-plane call at startup, on reload, or on a request.

The resolved identifiers live in the provider's encrypted settings blob as resolved_model and resolved_small_fast_model. A table keyed by ARN was tried and dropped: the server always writes the resolution as part of the same provider write, so it needs no separate lifecycle, migration, or clearing rules.

Resolution runs before the write, so a failed lookup rejects the save and stores nothing. Create resolves its own request, which carries the complete configuration. Update cannot: a PATCH supplies the model identifiers but inherits credentials from the stored row, so stored + patch is merged before the transaction to resolve against, then merged again inside it against the row being written. Both merges take the model identifiers from the patch, so they cannot disagree, and no AWS call happens inside a database transaction.

Reporting the failure at save time means an operator learns immediately that an ARN is wrong, deleted, or that the identity lacks bedrock:GetInferenceProfile, instead of discovering it later through a Bedrock 400 on adaptive-only models. Providers configured with plain model IDs store no resolution and make no AWS call, because those identifiers are already model identities.

Resolution uses the provider's own credential configuration: static access keys when set, otherwise the AWS default credential chain, and the role from role_arn when configured. An AIP ARN embeds the account ID, so resolving under a different principal cannot return another account's profile; it can only fail with a permission error, which surfaces as a rejected write.

Using an AIP ARN now requires coderd to reach bedrock.{region}.amazonaws.com. Deployments where only the gateway has AWS egress cannot configure AIP ARNs, and they find out when they save.

Implementation plan

Goal

Resolve Bedrock AIP ARNs once, when a provider is created or updated, and persist the result. The gateway reads the resolved model from the provider payload and makes no AWS call on the boot, reload, or request paths.

Identity

coderd resolves for every credential mode. An AIP ARN carries the account ID, so resolving under a different principal either succeeds with the same answer or fails with a permission error. The risk is a false negative, not a wrong answer.

  • Static access keys, with or without role_arn. The settings fully determine the identity, so coderd reaches the same principal the gateway will.
  • AWS default credential chain. coderd resolves under its own environment, which may legitimately lack bedrock:GetInferenceProfile even where the gateway has it. The save then reports an actionable message.

Behavior at write time

  1. Skip when neither model nor small_fast_model is an AIP ARN. This is the common case and costs nothing.
  2. Otherwise resolve both identifiers with a short deadline, before the provider write.
  3. On success, store the resolved identifiers in the same provider write and publish normally.
  4. On failure, surface the AWS error and store nothing. The remedy is to fix the ARN, configure static access keys, or grant bedrock:GetInferenceProfile.
  5. Re-resolve on every save. Bedrock has no UpdateInferenceProfile, so an ARN means one model forever, but re-resolving proves this provider's own identity can read the profile.

Gateway

ResolvedModelWithFallback and ResolvedSmallFastModelWithFallback return the stored resolution when present and the configured identifier otherwise. The fallback covers plain model IDs and providers saved before this change, degrading to pre-#28877 behavior rather than dropping the provider from the pool.

Out of scope

  • Normalizing directly configured system inference profile ARNs.
  • Provisioned throughput and prompt router identifiers.
  • Any gateway-side resolution, including a lazy fallback.
  • Displaying the resolved model in the UI.

Relates to https://linear.app/codercom/issue/AIGOV-488

Created by Coder Agents on behalf of @evgeniy-scherbina.

@linear-code

linear-code Bot commented Sep 8, 2026

Copy link
Copy Markdown

AIGOV-488

@github-actions

github-actions Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

@evgeniy-scherbina
evgeniy-scherbina force-pushed the yevhenii/aigov-488-resolve-inference-profiles-at-write-time branch from 3dfda05 to 5a04d32 Compare September 9, 2026 16:08
evgeniy-scherbina and others added 2 commits September 10, 2026 11:22
…#29175)

Follow-up to #29112, which stores the resolved Bedrock model in its own
table. Keyed by inference profile ARN, that table needed a migration,
three queries, dbauthz wrappers, a store-interface method, and a join in
the provider payload. This PR keeps the values in the settings blob
instead, where the rest of the provider's Bedrock configuration already
lives.

The plumbing that disappears is the point: `aibridgedserver` no longer
collects identifiers, queries mappings, or threads a map through
`aiProviderToProto`; it reads two fields off the settings it already
decoded. Net 314 deletions against 119 insertions, and no schema change.

Resolution still runs after the write commits, so no AWS call happens
inside a transaction, and still runs on every save. Storing the result
now means a second `UpdateAIProvider` with the resolved settings, which
is the one thing the table did not need.

Server ownership is enforced by clearing rather than validating. The
write path zeroes `resolved_model` and `resolved_small_fast_model`
before storing and rewrites them after resolution, so a client-supplied
value is discarded rather than rejected, and a stale value cannot
survive a settings change. That is one two-line helper instead of the
validation, merge carry-forward, and compare-before-write that an
earlier attempt at this needed.

What is lost relative to the table: two providers configured with the
same ARN each store their own copy, and each resolves it separately.
Resolution already ran per save, so this costs storage rather than AWS
calls.

Behavior is unchanged. An unresolved ARN still serves as its own
identity, a failed lookup still reports 400 while leaving the provider
stored, and the gateway still never calls the Bedrock control plane.

Relates to https://linear.app/codercom/issue/AIGOV-488

Created by Coder Agents on behalf of @evgeniy-scherbina.
…29179)

Follow-up to #29112, which writes the provider first and stores the
resolution in a second update. That leaves a window where a provider
exists with an unresolved profile ARN, which then has to be explained
everywhere: the gateway serves the ARN as its own identity, the audit
entry records a pre-resolution row, and the publish is skipped so the
provider only reaches the gateways after a restart.

Resolving before the write removes the state instead of describing it. A
failed lookup now rejects the save and stores nothing, on create and on
update alike.

Create resolves its own request, since a create carries the complete
configuration. Update cannot: a `PATCH` supplies the model identifiers
but inherits credentials and the external ID from the stored row, so the
config to resolve is stored + patch. That merge now lives in
`lookupAndMergeSettings` and is called twice, once before the
transaction to resolve against and once inside it against the row being
written. The two cannot disagree on the model identifiers, because both
take them from the patch, so no reconciliation or conflict handling is
needed.

The server-owned STS external ID is still generated inside the
transaction, as before. A value generated for the preview would be one
the operator's role trust policy could not reference yet either way, so
resolution assumes the role with the stored value.

Net effect against the base branch: the second `UpdateAIProvider` is
gone, `clearBedrockModelResolution` is gone (the resolution result is
always assigned, which discards anything a client sent), and the update
transaction is shorter than before this PR series started.

Relates to https://linear.app/codercom/issue/AIGOV-488

Created by Coder Agents on behalf of @evgeniy-scherbina.
@evgeniy-scherbina evgeniy-scherbina changed the title feat(coderd): resolve bedrock inference profiles when a provider is written feat: resolve bedrock inference profiles when a provider is written Sep 10, 2026
@evgeniy-scherbina
evgeniy-scherbina marked this pull request as ready for review September 10, 2026 17:07
@evgeniy-scherbina

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review

coder-agents-review Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Chat: Review in progress (16/16 reviewers complete) | View chat
Requested: 2026-09-10 17:07 UTC by @evgeniy-scherbina

deep-review v0.9.0 | Round 1 | 150c680..11ba473

Last posted: Round 1, 16 findings (1 P1, 2 P2, 5 P3, 3 Nit, 5 Note), REQUEST_CHANGES. Review

Finding inventory

Finding inventory - PR #29112

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 Note Open coderd/ai_providers_bedrock.go:49 All resolution failures (incl. transient AWS) return HTTP 400 with ARN/permission message R1 Netero, Leorio Yes
CRF-2 Note Open aibridge/provider/bedrock_inference_profile.go:119 Dedup branch for identical Model/SmallFastModel ARN is untested R1 Netero, Bisky Yes
CRF-3 P1 Open coderd/ai_providers.go:192 Create resolves (outbound AWS under coderd identity) before RBAC create authz: confused deputy, error oracle, DoS R1 Kurapika Yes
CRF-4 P3 Open coderd/ai_providers.go:332 Update resolves after only read authz: read-but-not-update actor drives coderd-identity AWS call + 400 detail leak R1 Kurapika Yes
CRF-5 P2 Open coderd/ai_providers_migrate.go:100 SeedAIProvidersFromEnv bypasses resolution: env-seeded AIP ARN providers store no resolution and degrade permanently R1 Ryosuke Yes
CRF-6 P2 Open coderd/ai_providers.go:332 Update resolves under stored (empty) external ID vs create; correct add-role+AIP config rejected with misleading diagnostic R1 Pariston P2, Hisoka Note, Mafuuu Note, Melody Note Yes
CRF-7 P3 Open aibridge/config/config.go:89 Unresolved/pre-existing AIP-ARN providers silently degrade on upgrade: no backfill, no signal, keys pricing/metrics/capabilities off raw ARN R1 Hisoka P3, Mafuuu P3, Chopper P3, Meruem P3, Pariston Note Yes
CRF-8 P3 Open coderd/ai_providers.go:897 Server-owned ResolvedModel invariant enforced by downstream overwrite, not the merge; future writer could let client spoof model identity R1 Zoro P3, Meruem Note, Ryosuke Note, Kurapika Note, Melody Note Yes
CRF-9 P3 Open coderd/ai_providers.go:192 Create now resolves before insert: RoleARN+AIP+external-ID-required trust policy cannot bootstrap (row never stored, external ID never surfaced) R1 Meruem Yes
CRF-10 P3 Open coderd/ai_providers_bedrock.go:22 Region-less Bedrock+AIP provider passes validation but resolution hardcodes baseURL="", rejecting the write with an unreachable remedy R1 Mafuuu Yes
CRF-11 Nit Open aibridge/config/config.go:89 New accessors ResolvedModel/SmallFastModelWithFallback lack doc comments and hand-roll a fallback cmp.Or covers R1 Gon, Leorio, Ging-Go Yes
CRF-12 Nit Open cli/aibridged.go:339 bedrockConfig is now a pure pass-through wrapper: inline/delete or fix the body-restating doc R1 Zoro, Gon Yes
CRF-13 Nit Open coderd/ai_providers.go:460 No-settings-path comment is detached from the code (the if req.Settings gate) it explains R1 Gon Yes
CRF-14 Note Open coderd/ai_providers.go:332 Update resolution (AWS call) runs before type-mismatch/external-id validations, so a misdirected PATCH gets a masking resolution error R1 Chopper Yes
CRF-15 Note Open coderd/aibridge/bedrock.go:19 Moved BedrockConfig mapping is tested only indirectly from cli; tests should move next to the code R1 Zoro Yes
CRF-16 Note Open coderd/ai_providers_bedrock.go:15 Same operation named resolveBedrockProfiles (coderd) vs ResolveBedrockModels (aibridge); identity paragraph duplicated verbatim R1 Gon Yes

Contested and acknowledged

None this round.

Round log

Round 1

Netero first pass: 0 P0-P3, 2 Notes (CRF-1, CRF-2); mechanical floor clean, panel proceeded. Panel of 15 (bisky, hisoka, mafu-san, mafuuu, pariston, chopper, ging-go, gon, leorio, kurapika, ryosuke, komugi, melody + wildcards meruem, zoro). Result: 1 P1, 2 P2, 4 P3, 3 Nit, 6 Note. Convergence on the external-ID/role-assumption ordering (CRF-6, CRF-9) and the unresolved-ARN silent-degradation cluster (CRF-5, CRF-7, CRF-8). Kurapika's authz-before-side-effect P1 (CRF-3/CRF-4) verified against code. Event: REQUEST_CHANGES. Reviewed against 150c680..11ba473.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@coderagents

coderagents Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Documentation Check

Updates Needed

  • docs/ai-coder/ai-gateway/providers.md (section Application inference profiles) - This PR moves application inference profile (AIP) ARN resolution from the gateway to coderd, performed once when a provider is created or updated. The section at the current PR head is back to the pre-PR text and is now inaccurate on three points:
    • It says "AI Gateway ... internally resolving and using the underlying model identity". Resolution now happens in coderd at provider write time; the gateway reads the stored resolution and makes no Bedrock control-plane call at startup, on reload, or on a request.
    • It says "the AWS identity used by the gateway must have bedrock:GetInferenceProfile permission". The identity that needs bedrock:GetInferenceProfile is now coderd's (the provider's own credential configuration used at save time), not the gateway's.
    • It says "If resolution fails, the provider is skipped." Resolution now runs before the write, so a failed lookup rejects the save with an actionable error and stores nothing, rather than silently dropping the provider from the pool.

    ⚠️ The docs change was reverted: this PR no longer contains any docs/ changes, so the section is byte-for-byte what main has. All three inaccuracies above remain.

  • docs/ai-coder/ai-gateway/providers.md - Consider documenting the new deployment constraint: configuring an AIP ARN now requires coderd to reach bedrock.{region}.amazonaws.com. Deployments where only the gateway has AWS egress cannot configure AIP ARNs.

Automated review via Coder Agents

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a well-reasoned relocation of Bedrock AIP resolution off the gateway's boot/reload/request path and onto the coderd write path, and the PR description is unusually honest about its tradeoffs (coderd now needs control-plane egress; providers saved before this change degrade to pre-#28877 behavior). The test suite is real: it drives the AWS SDK against a mock control plane and asserts persisted values, failure rollback, idempotency, and that a client-supplied resolution is discarded. Test density is 64.3%.

The panel found one blocker. Moving the AWS control-plane call into the create/update handlers introduced a confused-deputy authorization gap: resolution runs under coderd's own AWS identity before any RBAC check, so an unprivileged authenticated user can drive it (CRF-3). That is the reason for REQUEST_CHANGES.

The rest clusters into two themes worth reading together. First, the external-ID/role-assumption ordering: create ensures the server-owned external ID before resolving, update does not, so an add-role-plus-AIP PATCH resolves under an empty external ID and a correct config can be rejected (CRF-6), and moving resolution before the insert removes the persist-then-surface bootstrap for the external ID entirely (CRF-9). Second, unresolved ARNs stored without a signal: env seeding never resolves (CRF-5, permanent for that path), pre-existing rows have no backfill and no warning on upgrade (CRF-7), and the server-owned invariant lives in a downstream overwrite rather than the merge (CRF-8). None of these is fatal on its own, but each stores an ARN that the gateway then keys pricing, usage, capability detection, and metrics off of, silently.

Severity count: 1 P1, 2 P2, 4 P3, 3 Nit, 6 Note.

Several findings (CRF-5, CRF-7, CRF-9) need a human decision rather than a silent default: backfill, a release note, a warning log, or an explicit accepted-limitation. There is no assumed follow-up PR, so please decide them here.

Process note: the title CI job (102963864891) is red. The current title conforms to the repo's own validator (feat: with no scope short-circuits to valid per .github/workflows/contrib.yaml:172-174), so this is likely a stale run from an earlier title, but confirm it green rather than merging past it.

Hisoka, on the cross-account resolution concern: "Came looking for a split-brain between the two resolutions and found the author had already closed it. ♥"


coderd/ai_providers_migrate.go:100

P2 [CRF-5] Resolution is attached to the two HTTP handlers, not to the shared provider-write boundary, so the third writer (env seeding) stores AIP ARNs with no resolution and the gateway silently degrades. (Ryosuke)

SeedAIProvidersFromEnv inserts rows directly with codersdk.AIProviderSettings{Bedrock: dp.Bedrock} and never resolves; grep for resolveBedrockProfiles/ResolveBedrockModels returns zero hits in that file.

Verified: SeedAIProvidersFromEnv encodes from dp.Bedrock (line 100) and inserts (line 149) with no resolution. An operator who sets CODER_AI_GATEWAY_PROVIDER_N_BEDROCK_MODEL to an AIP ARN gets a stored row with ResolvedModel == "". Unlike CRF-7 (upgrade, heals on next API save), this path never heals: even the drift re-insert bypasses resolution, so the ARN string permanently becomes the capability predicate input and the Prometheus/pricing key. Before this PR, #28877 resolved at gateway construction, covering env-seeded providers. Fix: resolve at the shared write boundary all three writers pass through, or resolve in the seeder before encodeAIProviderSettings. If startup must not call AWS, then the seeder must reject an AIP ARN with an actionable error rather than store one that degrades silently — a human call.

🤖

coderd/ai_providers.go:897

P3 [CRF-8] mergeAIProviderSettings copies the server-owned ResolvedModel/ResolvedSmallFastModel verbatim from the patch, so the "a client cannot set them" invariant lives in a later overwrite instead of at the merge. (Zoro P3, Meruem Note, Ryosuke Note, Kurapika Note, Melody Note)

merged := *patch.Bedrock (line 897) carries the client-supplied resolved fields straight through. The same function pins the other server-owned field explicitly one line later: merged.ExternalID = existing.Bedrock.ExternalID (line 908) ... ResolvedModel gets neither.

Today no client can poison the resolution, because applyBedrockResolution unconditionally overwrites both fields and ResolveBedrockModels keys off Model/SmallFastModel, never ResolvedModel. But the safety of a server-owned field — the pricing, usage, capability, and metrics key — depends on a downstream overwrite always running, not on the merge that produces the stored value. A future write path that merges settings and forgets the overwrite lets a client inject a fake model identity into billing and metrics. The ExternalID handling in this same function is the codebase's own pattern for this class of field; match it by clearing the resolved fields in the merge (they are always recomputed):

merged := *patch.Bedrock
merged.ResolvedModel = ""
merged.ResolvedSmallFastModel = ""

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/ai_providers.go
// Resolve application inference profile ARNs before storing them, so an
// unresolvable profile is never written and the gateway never calls the
// Bedrock control plane.
resolved, err := resolveBedrockProfiles(ctx, req.Settings)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 [CRF-3] Create resolves an AIP ARN (an outbound AWS control-plane call under coderd's own identity) before any create authorization runs, so any authenticated user with no permission on ResourceAIProvider can drive it. (Kurapika)

The create handler performs no rbac authorization of its own. The only create check is authorizeContext(ActionCreate, ResourceAIProvider) inside q.InsertAIProvider, which runs in the InTx block at coderd/ai_providers.go:213, after resolution.

Verified: resolveBedrockProfiles runs at line 192; the only RBAC gate is inside InsertAIProvider within the InTx at line 213. A member can POST {"settings":{"bedrock":{"region":"...","model":"arn:aws:bedrock:...:application-inference-profile/..."}}} with no keys, and coderd calls GetInferenceProfile (and with role_arn, AssumeRole) under its own IRSA/instance identity against an attacker-controlled ARN and region. Three consequences, all before the permission check: an error oracle (the 400 leaks the raw AWS error via Detail in writeAIProviderResolutionError, distinguishing existence vs permission over coderd's account), availability (up to the 30s resolution timeout per unprivileged request), and driven sts:AssumeRole against an attacker-named role. Fix: authorize ActionCreate on ResourceAIProvider before ensureBedrockExternalID/resolveBedrockProfiles. The side effect, not just the DB write, must sit behind the gate.

🤖

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: a417d6a

Comment thread coderd/ai_providers.go
writeAIProviderError(ctx, api.Logger, rw, err, "update AI provider", "Internal error updating AI provider.")
return
}
resolved, err = resolveBedrockProfiles(ctx, preview)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-4] Update resolves against AWS after only a read authorization, so an actor with read-but-not-update on ResourceAIProvider triggers the same coderd-identity AWS call and 400 detail leak. (Kurapika)

The outside-transaction lookupAndMergeSettings -> lookupAIProvider -> GetAIProviderByID requires ActionRead ... Update permission is only enforced later, inside the transaction.

Same root cause as CRF-3, one gate weaker. Lower severity because it requires an existing provider and read access, but the same authorize-before-side-effect fix (authorize the update action before resolveBedrockProfiles at line 332) closes both.

🤖

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed: a417d6a

Comment thread coderd/ai_providers.go
writeAIProviderError(ctx, api.Logger, rw, err, "update AI provider", "Internal error updating AI provider.")
return
}
resolved, err = resolveBedrockProfiles(ctx, preview)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 [CRF-6] The update path resolves the profile under the stored external ID, which is empty when the same PATCH first adds role_arn, so a correct config can be rejected under a trust policy that requires the external ID. (Pariston P2, Hisoka Note, Mafuuu Note, Melody Note)

Create runs ensureBedrockExternalID(&req.Settings) (line 187) before resolveBedrockProfiles (line 192) ... Update does the opposite: it resolves at line 332 against preview ... ensureBedrockExternalID is not called on preview; it runs only inside the transaction at line 366, after resolution.

Verified against the diff. When an admin adds role_arn and an AIP ARN in one PATCH to a provider that had no role, the stored external ID is "", so resolution's AssumeRole runs with no ExternalId. AWS recommends an sts:ExternalId condition on cross-account trust policies (confused-deputy protection), and Coder generates the external ID specifically to satisfy it. If the trust policy enforces it, resolution fails and the save is rejected with a message pointing at the ARN/permissions, neither of which is the cause. Fix: call ensureBedrockExternalID(&preview) before resolveBedrockProfiles(ctx, preview), mirroring create. Not exercised by tests: the update subtests use bedrockSettings without role_arn. See CRF-9 for the deeper bootstrap tension this fix does not fully resolve.

🤖

@evgeniy-scherbina evgeniy-scherbina Sep 10, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a customer wants to set a RoleARN, require an ExternalID in the role's trust policy, and use that role to resolve an AIP, the setup is inherently sequential:

  1. Set RoleARN with a plain model ID.
  2. Read the generated ExternalID.
  3. Add it to the role's trust policy.
  4. Set the AIP.

If the trust policy does not require an external ID, setting the role and AIP together already works: resolution assumes the role without one, and AWS permits that. Calling ensureBedrockExternalID(&preview) does not solve the enforcing-policy case because the customer cannot have configured the newly generated value yet.

We could improve this UX later, but I would wait for customer signal.

Comment thread aibridge/config/config.go
return c.Protocol
}

func (c AWSBedrock) ResolvedModelWithFallback() string {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-7] Removing gateway-side resolution while leaving every stored row's ResolvedModel empty means existing AIP providers silently degrade on upgrade, with no backfill and no operator signal. (Hisoka P3, Mafuuu P3, Chopper P3, Meruem P3, Pariston Note)

ResolvedModelWithFallback returns Model when ResolvedModel == "", and every row that exists before this PR has ResolvedModel == "". ... capability detection, pricing, usage, and metrics all key off the ARN string instead of the model ID. The PR's own stated failure mode returns: a Bedrock 400 on adaptive-only models.

Strong convergence from five reviewers. The fallback collapses two distinct states — "plain model ID, no resolution needed" and "AIP ARN never resolved" — into "serve the configured string." For the second, capability/pricing/usage/metrics key off the raw ARN and adaptive-only models revert to Bedrock 400s indefinitely until an operator re-saves each provider. Severity held at P3 (not higher) because #28877 is in no release tag (git tag --contains empty), so the exposed population is main/dogfood rows created between the two merges, and the state self-heals on any re-save. Blast radius narrow, teeth real. This needs a human decision, not a silent default: a first-write backfill, a release note telling operators to re-save AIP providers, or at minimum a warning log at gateway construction when a served Bedrock provider's identifier is an AIP ARN with empty resolution. ResolvedModelWithFallback already has the type in hand to tell the two cases apart via isApplicationInferenceProfileARN(c.Model).

🤖

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous functionality hadn't been released yet, so it should be okay.

Comment thread coderd/ai_providers.go
// Resolve application inference profile ARNs before storing them, so an
// unresolvable profile is never written and the gateway never calls the
// Bedrock control plane.
resolved, err := resolveBedrockProfiles(ctx, req.Settings)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3 [CRF-9] Moving the first role assumption to create time removes the persist-then-fix bootstrap for an AIP-ARN provider that assumes a role under a server-generated external ID. (Meruem)

Pre-PR, create called ensureBedrockExternalID then InsertAIProvider with no AWS call ... the row was always persisted and the generated external ID was returned, so an operator could read it, add sts:ExternalId to their role trust policy, and let the gateway assume the role later. Post-PR, create resolves before the insert and rejects the write on AssumeRole failure, so the provider is never stored and the generated external ID is never surfaced.

This is the deeper tension behind CRF-6: an operator who follows AWS guidance and requires the external ID in the trust policy cannot satisfy it, because the value they must trust is only knowable from a create that now cannot succeed. Bounded to RoleARN + AIP ARN + a trust policy that enforces the external ID. There is an escape (supply your own external ID on create, which create accepts), but it is undocumented and contradicts the "server-owned, server-generated" framing. Decide it explicitly: document the required client-supplied external ID for this combination, or persist the row on resolution failure with the error attached so the external ID is discoverable, or accept the limitation.

🤖

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same bootstrap limitation, but on create. The supported sequence is:

  1. Create the provider with RoleARN and a plain model ID.
  2. Read the generated ExternalID.
  3. Add it to the role�s trust policy.
  4. Update the provider to use the AIP.

Creating with both RoleARN and an AIP works only when the trust policy does not require an external ID. Calling ensureBedrockExternalID before resolution cannot solve the enforcing-policy case because the customer cannot trust a value that has not been returned yet.

I think we should accept this limitation for now and improve the UX if there is customer demand.

Comment thread coderd/ai_providers.go
writeAIProviderError(ctx, api.Logger, rw, err, "update AI provider", "Internal error updating AI provider.")
return
}
resolved, err = resolveBedrockProfiles(ctx, preview)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [CRF-14] On update, resolution (an AWS call) runs before the type-mismatch and external-ID validations, so a misdirected PATCH can surface a resolution error that masks the real problem. (Chopper)

An operator who PATCHes Bedrock settings onto a non-Bedrock provider, using an ARN that does not resolve, gets the 400 "Could not resolve the Bedrock application inference profile" instead of "bedrock settings are only valid for type=anthropic or type=bedrock."

It also spends an AWS call on a request the transaction will reject regardless. Niche, but it is a diagnostic mis-signal for exactly the kind of operator mistake error messages exist to catch. Related to the authz ordering in CRF-4.

🤖

// endpoint and is always non-empty, so it cannot serve as a Bedrock detection
// signal; gate on the settings alone via
// [codersdk.AIProviderBedrockSettings.IsConfigured].
func BedrockConfig(baseURL string, bedrock *codersdk.AIProviderBedrockSettings) *aibridgeconfig.AWSBedrock {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [CRF-15] The settings-to-config mapping moved into coderd/aibridge, but its only tests stayed in cli/aibridged_internal_test.go and reach it through the bedrockConfig wrapper. (Zoro)

coderd/aibridge has no _test.go file, so BedrockConfig (the shared mapping now used by both the gateway build path and the write-path resolver) is exercised only indirectly from the cli package.

If the CRF-12 wrapper is removed, move the BedrockSettingsPresent/BedrockSettingsEmpty cases next to the code they now own in coderd/aibridge/bedrock_test.go.

🤖

// identity and misshape every request made through it.
func (api *API) writeAIProviderResolutionError(ctx context.Context, rw http.ResponseWriter, err error) {
api.Logger.Warn(ctx, "resolve bedrock inference profile", slog.Error(err))
httpapi.Write(ctx, rw, http.StatusBadRequest, codersdk.Response{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [CRF-1] Every resolution failure returns HTTP 400 with a "check the ARN / grant bedrock:GetInferenceProfile" message, including transient AWS failures (timeout, throttling, 5xx, missing egress). (Netero, Leorio)

A Bedrock throttle, a control-plane 5xx, or the 30s deadline expiring (coderd lacks egress) then reports a client input error. The raw cause is preserved in Detail, and rejecting the write on failure is the PR's intended design.

The wording itself is otherwise sound (states the failed thing, names the remedy, raw cause rides in Detail); the misdiagnosis is only for the transient-failure subset. Worth knowing, not worth blocking on.

🤖

defer cancel()

for _, profileARN := range profiles {
if _, ok := resolved[profileARN]; ok {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [CRF-2] The dedup branch (if _, ok := resolved[profileARN]; ok { continue }) is untested. (Netero, Bisky)

ResolveBedrockModels reads 90.0% covered; no test configures the same profile ARN for both Model and SmallFastModel, so the branch that avoids a second identical lookup never executes under test.

Low value: a one-line guard with obvious behavior. Noted for completeness.

🤖

"github.com/coder/coder/v2/coderd/httpapi"
"github.com/coder/coder/v2/codersdk"
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note [CRF-16] The same operation is named resolveBedrockProfiles in coderd and ResolveBedrockModels in aibridge, and the identity paragraph is duplicated verbatim across ResolveBedrockModels and resolveInferenceProfile. (Gon)

resolveBedrockProfiles calls provider.ResolveBedrockModels; both take Bedrock settings and return the identical ARN-to-model map. "Profiles" vs "Models" for one operation across the call boundary is a mild readability cost.

Each name is locally defensible (coderd resolves profiles, the provider returns models), so this is an observation, not a required change. If the duplicated identity sentence drifts, the two copies will disagree; having the internal helper reference the exported contract avoids that.

🤖

Comment thread cli/aibridged.go Outdated
// the generic upstream endpoint and is always non-empty, so it cannot
// serve as a Bedrock detection signal; gate on the settings alone via
// [codersdk.AIProviderBedrockSettings.IsConfigured].
// bedrockConfig is [agplaibridge.BedrockConfig], shared with the provider

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agree, looks redundant.

if bedrock == nil {
return nil
}
settings := *bedrock

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: manual dereference smells a bit

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, it was already like this before the PR. I just moved this code into a shared package.

Comment thread coderd/ai_providers.go
// Provider configuration has side effects outside the database, notably
// the Bedrock profile lookup below, so the permission is checked before
// any of them rather than only by dbauthz on the write.
if !api.Authorize(r, policy.ActionCreate, rbac.ResourceAIProvider) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

Comment thread coderd/ai_providers.go Outdated
)
err := api.Database.InTx(func(tx database.Store) error {
old, err := lookupAIProvider(ctx, tx, idOrName)
old, existing, err := lookupAndMergeSettings(ctx, tx, idOrName, req.Settings)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit old, existing -> old, merged

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed: 9d5979b

@evgeniy-scherbina
evgeniy-scherbina enabled auto-merge (squash) September 11, 2026 15:58
@evgeniy-scherbina
evgeniy-scherbina merged commit b405d57 into main Sep 11, 2026
28 checks passed
@evgeniy-scherbina
evgeniy-scherbina deleted the yevhenii/aigov-488-resolve-inference-profiles-at-write-time branch September 11, 2026 16:03
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 11, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants