Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
fix: single source of truth for bedrock provider validation #27686
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Uh oh!
There was an error while loading. Please reload this page.
fix: single source of truth for bedrock provider validation #27686
Changes from all commits
35ed65b20a46024375d2178ae956File filter
Filter by extension
Conversations
Uh oh!
There was an error while loading. Please reload this page.
Jump to
Uh oh!
There was an error while loading. Please reload this page.
There are no files selected for viewing
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it might be confusing to have two exported, very similar methods on the same struct:
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Regarding a single source of truth and unification: the runtime validation may need to be stricter than the SDK validation, because runtime validation runs last, after AI Gateway has enriched the configuration from environment variables (see buildBedrockCredentials):
This PR seems fine, but it's worth keeping in mind.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3 [CRF-5]
Validatere-derives "unknown protocol" by excluding two named constants, so the protocol list now lives in two places and a third protocol added toValidationErrorswill be reported as unknown. (Netero)Keep one enumeration: a
knownProtocol() boolhelper with the same switch, or haveValidationErrorsreport the unknown protocol as aFieldError{Field: "protocol"}and letValidatestop special-casing it. A PR whose thesis is single source of truth should not split an enumeration in the process.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P3 [CRF-6]
allowedBedrockFieldsis a literal copy of the codersdk JSON tags, so it cannot detect the drift its comment says it guards against, and the assertion it adds is already made two lines later. (Netero)A guard that works has to compare against the real tags, which means it belongs in a package that may import codersdk: a reflect-over-
json-tags test next toBedrockConfigFromSettingsincoderd/aibridge. As written, the comment claims more than the test proves, which is worse than no guard because the next reader trusts it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit [CRF-10] The 8-line
FieldErrortocodersdk.ValidationErrormapping is duplicated verbatim in the create and update handlers. (Netero)One helper (
bedrockValidations(baseURL string, s *codersdk.AIProviderBedrockSettings) []codersdk.ValidationError) removes the copy and gives the two handlers a single place to stay consistent.That divergence is CRF-1 and CRF-2. The helper is also where the
settings.prefix decision Copilot flagged should live, so the field-path fix lands in one place.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P1 [CRF-1] The update handler validates the incoming patch instead of the merged settings, so a PATCH that omits
base_urlskips bedrock validation entirely and persists the exact invalid row this PR exists to prevent. (Netero)Reproduced against head: create a valid invoke-model bedrock provider, then
returns 200 and the stored row reads
model="" small_fast_model="". At startup that row failsruntimeCfg.Validate()withmodel requiredand never enters the route table, which is the "silently skipped, every chat 404s" failure in the PR description.Validate after the merge, inside the transaction, against
ptr.NilToDefault(req.BaseURL, old.BaseUrl). That also removes the requirement that callers resend fields they are not changing.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2 [CRF-2] Gating validation on
BedrockConfigFromSettingsreturningokmeans a patch that clears every load-bearing field escapes validation, becauseIsConfigured()is false for the cleared blob. (Netero)Reproduced against head: PATCH with
BaseURLset andSettings.Bedrock = &codersdk.AIProviderBedrockSettings{}returns 200 and the stored row readsregion="" model="" small_fast_model="". Same runtime consequence as CRF-1: atype=bedrockrow thatbuildProviderrefuses.The gate is protecting a different case (a
type=anthropicprovider carrying an unconfigured bedrock blob). Validating the merged result narrows it correctly: after the merge, a bedrock-typed row must satisfy the bedrock rules regardless of what the patch contained.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit [CRF-7]
cdaibridgeis a third alias forcoderd/aibridgein a tree that already standardized onagplaibridgefor this exact collision. (Netero)This PR uses
agplaibridgeincli/aibridged.goandcdaibridgehere. Useagplaibridgein both.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit [CRF-9] The validation loop ranges over the
outmap, so with two invalid bedrock providers the returned error names an arbitrary one, immediately above the comment that establishes deterministic ordering for exactly this reason. (Netero)Fold the check into the sorted
resloop below, or sort the names first. An operator who fixes the named provider and restarts should not be told about a different one at random.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
P2 [CRF-4] The new seed guard misses base-URL-only bedrock providers, because the seed decides a provider is bedrock with
codersdk.IsBedrockConfigured(base URL counts) while the guard gates onBedrockConfigFromSettings(base URL does not count). (Netero)Reproduced against head on postgres: seeding
AIProviderConfig{Type: "bedrock", Name: "baseurl-only", BedrockBaseURL: "https://bedrock-runtime.us-east-1.amazonaws.com"}returns no error and persistssettings={"_type":"bedrock","_version":1}. No model, no small fast model, sobuildProviderrefuses the row and every chat routed at it 404s: the class the guard's own comment claims to close.This also contradicts the contract documented on
IsBedrockConfigured(codersdk/aiproviders_bedrock.go:122-129): "the seed, the runtime config builder, and the legacy validator must all agree on what counts as a Bedrock provider." Gate the guard on the same predicate the seed used:dp.Bedrock != nilis already the loop condition, so drop theokearly-continue and validate the converted config.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nit [CRF-8]
cfg, ok := cdaibridge.BedrockConfigFromSettings(...)shadows thecfg codersdk.AIBridgeConfigparameter ofprovidersFromEnv. (Netero)Rename the local to
bedrockCfg, which is whatcli/aibridged.go:285calls it.Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.