-
Notifications
You must be signed in to change notification settings - Fork 1.5k
feat: add experimental CLI to price unpriced AI models #27926
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
6f6b074
798058b
c23c21d
69a21e3
2c6d22c
700d241
9d257a6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,6 +6,7 @@ import ( | |
| "context" | ||
| _ "embed" | ||
| "encoding/json" | ||
| "sync" | ||
|
|
||
| "golang.org/x/xerrors" | ||
|
|
||
|
|
@@ -60,3 +61,32 @@ func parseSeed(data []byte) ([]seedRow, error) { | |
| } | ||
| return rows, nil | ||
| } | ||
|
|
||
| // modelKey identifies a priced model. | ||
| type modelKey struct { | ||
| provider string | ||
| model string | ||
| } | ||
|
|
||
| // defaultPricedModels indexes the embedded price book by provider and model. | ||
| // Built on first use, since a deployment that never sets a price never needs | ||
| // it. | ||
| var defaultPricedModels = sync.OnceValue(func() map[modelKey]struct{} { | ||
| rows, err := parseSeed(seedJSON) | ||
| if err != nil { | ||
| panic(xerrors.Errorf("parse embedded price seed: %w", err)) | ||
| } | ||
| index := make(map[modelKey]struct{}, len(rows)) | ||
| for _, row := range rows { | ||
| index[modelKey{provider: row.Provider, model: row.Model}] = struct{}{} | ||
| } | ||
| return index | ||
| }) | ||
|
|
||
| // IsDefaultPriced reports whether the embedded price book already carries a | ||
| // price for the model. Coder owns those prices and re-applies them on every | ||
| // startup, so an operator price set for one would not survive a restart. | ||
| func IsDefaultPriced(provider, model string) bool { | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Initially, I thought we would need to restore the models removed between 2.35 and 2.36. However, I think this is not required. After the patch lands, the CLI will only be available after 2.36, meaning that whatever was released on the price book is what is considered a model with "default" source (once AIGOV-589 lands), meaning that if operators decide to add custom values for removed models, these will be considered "custom". So we have 2 paths going forward:
Given that cost control was officially released in 2.36 and the cost reporting via the effective group is only available from 2.36 (even though raw interception cost without a group was released in 2.35, but all the data returned by the endpoints always guarantees the effective group constraint), and that it is unlikely that deprecated models will still be used, I would say option 1 is an acceptable approach. However, I think this brings up another topic on how we should handle "deprecated" models going forward: should we remove them from the price book? Because this would mean that for customers that skip a version, some models might be considered "unpriced", while for customers that do not skip any versions, they would be considered "default" (since we never delete models from the database). In conclusion, this would mean that we could end up with different customers with different default models in their databases. Wdyt?
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think it's acceptable for those models.
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Discussed internally and we decided that it is ok to not keep removed models in the price book. With the introduction of custom pricing, customers will be able to introduce models not built-in if needed. |
||
| _, ok := defaultPricedModels()[modelKey{provider: provider, model: model}] | ||
| return ok | ||
| } | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,28 @@ | ||
| package providers | ||
|
|
||
| import "github.com/coder/coder/v2/coderd/database" | ||
|
|
||
| // Supported lists the providers a model price may be set for. | ||
| // | ||
| // openai-compat is excluded: it is a generic passthrough, so the upstream | ||
| // vendor is unknown and a price cannot be attributed to it. Listed explicitly | ||
| // rather than derived from ai_provider_type so a new provider is opt-in. | ||
| var Supported = []database.AIProviderType{ | ||
| database.AIProviderTypeAnthropic, | ||
| database.AIProviderTypeAzure, | ||
| database.AIProviderTypeBedrock, | ||
| database.AIProviderTypeCopilot, | ||
| database.AIProviderTypeGoogle, | ||
| database.AIProviderTypeOpenai, | ||
| database.AIProviderTypeOpenrouter, | ||
| database.AIProviderTypeVercel, | ||
| } | ||
|
|
||
| // SupportedStrings returns the supported providers as plain strings. | ||
| func SupportedStrings() []string { | ||
| ids := make([]string, len(Supported)) | ||
| for i, provider := range Supported { | ||
| ids[i] = string(provider) | ||
| } | ||
| return ids | ||
| } |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Uh oh!
There was an error while loading. Please reload this page.