fix(coderd): require bedrock models when seeding providers from env - #27868
fix(coderd): require bedrock models when seeding providers from env#27868evgeniy-scherbina wants to merge 1 commit into
Conversation
538eadb to
cb312e6
Compare
|
/coder-agents-review |
|
Chat: Review posted | View chat Review history
deep-review v0.9.0 | Round 1 | Last posted: Round 1, 5 findings (1 P1, 2 P2, 1 P3, 1 Note), COMMENT. Review Finding inventoryFinding inventory, PR #27868Findings
Contested and acknowledgedNone. Law analysisNot run. Effective additions 92, below the 1000 threshold. Round logRound 1Netero-only first pass (P1 present, panel gated). 1 P1, 2 P2, 1 P3, 1 Note. Reviewed against 1c993c7..cb312e6. About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
First-pass review only. These are mechanical findings from a single first-pass reviewer; the full review panel has not reviewed this PR yet and will do so after these are addressed.
The validation itself is well placed: it runs before any database work, both call sites are covered by real tests (reverting the production change makes both new subtests fail, so they are not tautological), and the error strings carry provider identity plus remediation. Test density is 64%, and the added SmallFastModel fields in the five pre-existing subtests are needed to keep those tests on their original paths rather than padding.
The PR body asks whether this closes the gaps from #27846 and what remains. It closes one gap in a class of gaps, and the two most significant findings are about what it leaves open: deployments that already hit AIGOV-564 are not repaired by this change (CRF-1), and a base-URL-only indexed Bedrock provider still produces the same dead row and bare 404 (CRF-2).
Findings: 1 P1, 2 P2, 1 P3, 1 Note.
Netero on CRF-2: "Root cause is the class, not the instance: the seed's Bedrock predicate is wider than the set of rows the runtime can build, and this PR closes one gap in that gap set (missing models) while leaving the other."
🤖 This review was automatically generated with Coder Agents.
| // indexed ones carry no defaults, so a provider migrated from | ||
| // legacy to indexed env vars loses its models silently. | ||
| if err := validateSeededBedrockModels(bedrock); err != nil { | ||
| return nil, xerrors.Errorf("indexed AI provider %q: %w, set BEDROCK_MODEL and BEDROCK_SMALL_FAST_MODEL on it", name, err) |
There was a problem hiding this comment.
P1 [CRF-1] The remediation this error prints does not repair a Bedrock row that was already seeded without models, so startup goes green while the provider keeps returning 404. (Netero)
The drift hash deliberately excludes
ModelandSmallFastModel(canonicalAIProvider, lines 220-227). In thecase found:branch the seed compares hashes andcontinues on equality (lines 143-145); it never updatesSettings.
- Restart after this PR: startup fails with "set BEDROCK_MODEL and BEDROCK_SMALL_FAST_MODEL on it".
- Operator sets both env vars.
- Startup succeeds, the stored row is untouched with empty models,
buildProviderstill refuses it (cli/aibridged.go, bedrock branch), and every request to the provider still 404s.Verified: I inserted a Bedrock row with
{"region":"us-east-1"}settings, ranSeedAIProvidersFromEnvwith both models set in the indexed config, and asserted the stored settings afterwards.SeedAIProvidersFromEnvreturned nil andstored.Bedrock.Model/stored.Bedrock.SmallFastModelwere both still empty.
I verified both halves of the mechanism independently: canonicalAIProvider hashes only type, base URL, region, and keys hash, and the found branch returns early on hash equality without touching Settings.
This is the population the linked issue describes: indexed env vars have no model defaults, so any existing indexed Bedrock provider configured without BEDROCK_MODEL already has the broken row. For them the new error is a detour, not a fix, and the message actively misleads by naming env vars that cannot repair the row. Either include the models in the canonical hash so the stale row surfaces as drift, or detect the models-empty existing row and say the row must be fixed through the API.
🤖
| // Unlike the legacy CODER_AI_GATEWAY_BEDROCK_* options, the | ||
| // indexed ones carry no defaults, so a provider migrated from | ||
| // legacy to indexed env vars loses its models silently. | ||
| if err := validateSeededBedrockModels(bedrock); err != nil { |
There was a problem hiding this comment.
P2 [CRF-2] The same dead-row-and-404 failure still ships for an indexed Bedrock provider configured with BEDROCK_BASE_URL only; models are validated, credentials-or-region are not. (Netero)
IsBedrockConfigured(baseURL, b)is true on base URL alone, soCODER_AI_GATEWAY_PROVIDER_0_TYPE=bedrockplusBEDROCK_BASE_URL,BEDROCK_MODEL,BEDROCK_SMALL_FAST_MODEL(region and credentials from the AWS environment, the documented VPC/FIPS case inIsBedrockConfigured's own doc comment) passesReadAIProvidersFromEnv, passes the new model check, and is inserted. At runtimebedrockConfiggates onAIProviderBedrockSettings.IsConfigured(), which ignoresModel/SmallFastModel, returns nil, andbuildProviderrefuses the spec.Verified with a throwaway internal test in
cli:buildProvideron that exact spec returnsbedrock provider has no bedrock credentials configuredwhilecodersdk.IsBedrockConfiguredreports true for the same input.
Confirmed by reading both predicates: IsBedrockConfigured returns baseURL != "" || b.IsConfigured(), while bedrockConfig consults only IsConfigured(), whose doc comment lists region, role ARN, and access keys. The seed and the runtime disagree on what a buildable Bedrock provider is, and this PR narrows that disagreement by one field instead of closing it. Gating the seed on what the runtime actually requires (bedrockConfig(baseURL, settings) != nil plus config.AWSBedrock.Validate()) closes both this and the missing-model case with one check.
🤖
| // request. Without them the provider fails to build and is skipped, | ||
| // leaving every request to it to return a bare 404, so failing here | ||
| // keeps the dead row out of the database. | ||
| func validateSeededBedrockModels(b codersdk.AIProviderBedrockSettings) error { |
There was a problem hiding this comment.
P2 [CRF-3] validateSeededBedrockModels duplicates codersdk.validateAIProviderBedrockModels, added one commit earlier by the PR this follows up. (Netero)
codersdk/aiproviders.go:402already encodes "invoke-model requires Model and SmallFastModel", is called from bothCreateAIProviderRequest.ValidateandUpdateAIProviderRequest.Validate, and carries the same rationale in its doc comment ("would be skipped at gateway startup and every request to it would 404").git log -Sconfirms it landed in 1142706 (#27846), the immediate predecessor commit.The copy is protocol-unaware: it requires both models unconditionally, where the original skips the check for non-invoke-model protocols.
Verified: the codersdk validator returns early unless ResolvedProtocol() == invoke-model, and the new copy has no protocol branch. That is harmless today only because the indexed env key switch has no BEDROCK_PROTOCOL case, and nothing in either file records that dependency. The day a protocol env key is added, seeding rejects valid mantle providers. Export the codersdk validator (or a thin wrapper over it) and call it from both paths so the requirement and its protocol condition live in one place.
🤖
| // indexed ones carry no defaults, so a provider migrated from | ||
| // legacy to indexed env vars loses its models silently. | ||
| if err := validateSeededBedrockModels(bedrock); err != nil { | ||
| return nil, xerrors.Errorf("indexed AI provider %q: %w, set BEDROCK_MODEL and BEDROCK_SMALL_FAST_MODEL on it", name, err) |
There was a problem hiding this comment.
P3 [CRF-4] The indexed error names BEDROCK_MODEL, which is not an environment variable an operator can set. (Netero)
The real variables are
CODER_AI_GATEWAY_PROVIDER_<N>_BEDROCK_MODELandCODER_AI_GATEWAY_PROVIDER_<N>_BEDROCK_SMALL_FAST_MODEL(cli/server.go:3170, key switch at 3383). "on it" refers to a provider identified by name, but the env vars are indexed by number, so the operator has to map name back to index themselves. The sibling validator inReadAIProvidersFromEnvalready solves both problems: it reportsprovider %d (%s): ...with the index and it knows the active prefix (aiGatewayProviderEnvPrefixvsaiBridgeProviderEnvPrefix).
Confirmed: BEDROCK_MODEL is a key suffix reached only under the CODER_AI_GATEWAY_PROVIDER_<N>_ or CODER_AIBRIDGE_PROVIDER_<N>_ prefix, and the prefix in use is deployment-dependent. An operator following this message literally sets a variable that does nothing. Naming the index and the active prefix, or moving the check next to the existing BEDROCK_* consistency checks in ReadAIProvidersFromEnv, gives the correct name for free.
🤖
| // configured models into every upstream request. Both options | ||
| // carry defaults, so this only fires when an operator sets one | ||
| // to the empty string. | ||
| if err := validateSeededBedrockModels(bedrock); err != nil { |
There was a problem hiding this comment.
Note [CRF-5] The check turns a partially-broken deployment into a deployment that will not boot. (Netero)
SeedAIProvidersFromEnvfailure is fatal in both entry points (cli/server.go:1168,enterprise/cli/server.go:186). An operator running today with an indexed Bedrock provider and noBEDROCK_MODELhas one dead provider and a working coderd; after this change coderd refuses to start. That matches the existing fail-fast behavior of this file (drift already aborts startup), so I am not filing it as a finding, but the blast radius is worth stating in the PR body.
Agreed on both counts: consistent with the file's existing behavior, and worth stating in the PR body so whoever rolls this out knows the failure mode changed from one degraded provider to a refused startup. Note only, no change requested.
🤖
|
We decided to remove support for the LegacyEnv and IndexedEnv provider configurations, so closing this PR. |
Follow-up to: #27846
Related to: https://linear.app/codercom/issue/AIGOV-564/aibridge-bedrock-provider-skipped-404-on-all-routes-when-settings-omit
An env-seeded Bedrock provider always uses the InvokeModel protocol, which substitutes the configured models into every upstream request. When
BEDROCK_MODELorBEDROCK_SMALL_FAST_MODELis missing the provider fails to build and is skipped, so every request routed to it returns a bare 404 with nothing explaining why.Validate both model identifiers while seeding providers from the environment and fail startup with a descriptive error instead of writing a dead provider row. This applies to the legacy
CODER_AI_GATEWAY_BEDROCK_*options, where the defaults mean it only fires if an operator sets one to the empty string, and to the indexedCODER_AI_GATEWAY_PROVIDER_<N>_*options, which carry no defaults.For AGENTS: Review this in the context of #27846. Does it address the gaps introduced by that PR? Identify any remaining gaps or edge cases.