-
Notifications
You must be signed in to change notification settings - Fork 615
Open
Description
Currently, to override the default capabilities (like constrained: 'all') for a model using the openAICompatible plugin, one must implement a custom resolver that manually defines the model using defineCompatOpenAIModel and compatOaiModelRef.
This is verbose and inconvenient, especially when just trying to enable a capability that the model supports but isn't enabled by default in the generic model info (e.g. native structured output for Fireworks AI).
It would be much better if:
openAICompatibleplugin accepted acapabilitiesormodelInfooverride in its configuration.- Or if
ai.generaterespected theinfoproperty passed inmodelRefat runtime (currentlyresolveModelignores it).
Example of current workaround:
openAICompatible({
name: 'fireworks',
apiKey: process.env.FIREWORKS_API_KEY,
baseURL: '...',
resolver: async (client, actionType, actionName) => {
if (actionType === 'model') {
return defineCompatOpenAIModel({
name: actionName,
client,
modelRef: compatOaiModelRef({
name: actionName,
info: {
supports: {
multiturn: true,
media: true,
tools: true,
toolChoice: true,
systemRole: true,
constrained: 'all', // <--- The goal
},
},
}),
});
}
return undefined;
},
})Metadata
Metadata
Assignees
Labels
No labels
Type
Projects
Status
No status