Since we are using langchain, adding bedrock support is very simple:

pip install langchain_aws
Go to this function and add the following code:

elif config['type'].lower() == 'bedrock':
    from langchain_aws import ChatBedrock
    llm = ChatBedrock(
        model_id=config['name'],
        model_kwargs=dict(temperature=temperature),
    )
   return llm

In case you are using anthropic you should also add llm = AnthropicFunctions(llm=llm)
3. In the config_default.yml file, at each relevant definition of llm (when you want to change bedrock) you should change type: 'bedrock' and the name to the relevant model name.
4. If you are not using anthropic, as the meta-prompt (not recommended), you should also change:

meta_prompts:
    folder: 'prompts/meta_prompts_classification'

to:

meta_prompts:
    folder: 'prompts/meta_prompts_completion'

When I'll have access to bedrock, I'll upload a PR that adds it to the main code, meanwhile, if someone can verify that it's working and upload a PR it could be great.

Any plan to add AWS Bedrock suppport? #100

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions