Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[BUG] assume_model_exists by-passes Chat-object creation checks but not Chat#ask in Bedrock provider #366

@ken-brevio

Description

@ken-brevio

Basic checks

  • I searched existing issues - this hasn't been reported
  • I can reproduce this consistently
  • This is a RubyLLM bug, not my application code

What's broken?

I'm trying to access a model in Bedrock that is not specified in the ModelRegistry.

I assumed that I could use assume_model_exists but it doesn't really work.

How to reproduce

chat = RubyLLM.chat(
  provider: :bedrock,
  model: "mistral.pixtral-large-2502-v1:0",
  assume_model_exists: true
)

=>
#<RubyLLM::Chat:0x000000015b551908
...

chat.ask("What is your name?")
(llm-poc):85:in `<main>': Unknown model: mistral.pixtral-large-2502-v1:0 (RubyLLM::ModelNotFoundError)

Expected behavior

I expect the Chat#ask to at least hit the bedrock API but it doesn't look like it.

What actually happened

ModelNotFoundError is returned in stead

Environment

ruby: 3.3.5 (2024-09-03 revision ef084cc8f4) [arm64-darwin24]
RubyLLM: 1.6.4
Provider: Bedrock
OS: MacOS

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions