-
-
Notifications
You must be signed in to change notification settings - Fork 654
Closed
Labels
Description
For some applications it may be useful to provide API keys for models at runtime when a prompt is executed. That's not very easy right now - you can set model.key = "x" but that's then shared across all uses of that model instance. This is bad for multi-user environments like web applications, plus there's no easy way to create new model instances - llm.get_model(model_id) returns a single shared object.
I don't want to break existing code here, so I'm going to make this a new optional argument to model.prompt("prompt", key=...).