Thanks to visit codestin.com
Credit goes to github.com

Skip to content

feat(http-api-bindings): accept llmman model kinds - #4536

Open
ericcurtin wants to merge 1 commit into
TabbyML:mainfrom
ericcurtin:feat/llmman-model-kind
Open

feat(http-api-bindings): accept llmman model kinds#4536
ericcurtin wants to merge 1 commit into
TabbyML:mainfrom
ericcurtin:feat/llmman-model-kind

Conversation

@ericcurtin

Copy link
Copy Markdown

llmman is a local model runner that serves the Ollama API (alongside OpenAI- and Anthropic-compatible ones), so llmman/completion and llmman/embedding route to the same bindings as their counterparts.

[model.completion.http]
kind = "llmman/completion"
api_endpoint = "http://localhost:17434"
model_name = "qwen3.8"

Chat needs no kind of its own — llmman also serves an OpenAI-compatible API, and openai/chat already handles that. Adding a third alias there would have been noise.

To be straight about the scope: this is discoverability rather than new capability. api_endpoint is required for these kinds anyway, so pointing ollama/completion at port 17434 works today. The distinct kind just lets it be named for what it is in config.

Testing: cargo check -p http-api-bindings passes; cargo fmt --check clean. Not verified against a live server.

AI-assisted, reviewed before submitting.

llmman (https://github.com/llmmanorg/llmman) is a local model runner that
serves the Ollama API, so llmman/completion and llmman/embedding route to
the same bindings as their counterparts.

Chat needs no kind of its own: llmman also serves an OpenAI-compatible
API, which openai/chat already handles.

Signed-off-by: Eric Curtin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant