Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix(llm): correct LiteLLM embed() model override + finish #1593 review items#1626

Merged
itomek merged 3 commits into
mainfrom
fix/litellm-embed-model-override
Jun 12, 2026
Merged

fix(llm): correct LiteLLM embed() model override + finish #1593 review items#1626
itomek merged 3 commits into
mainfrom
fix/litellm-embed-model-override

Conversation

@itomek

@itomek itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Closes #1625

Why this matters

After #1593 added the LiteLLM provider, LiteLLMProvider.embed() crashed with TypeError: embedding() got multiple values for keyword argument 'model' the moment a caller overrode the model — model was passed both explicitly and inside the spread **call_kwargs, and that path had zero test coverage. Now embed() works with and without a model override, and the rest of the #1593 review is closed out: the provider is documented, the create_client docstring lists it, and the redundant drop_params handling is collapsed to a single per-call default callers can override.

Test plan

  • pytest tests/unit/test_litellm_provider.py → 12 passed (incl. 2 new embed() tests; the override test reproduced the TypeError before the fix)
  • util/lint.py --black --isort --flake8 → all PASS
  • CI green after maintainer approves the workflow run

embed() raised TypeError when a caller overrode the model: model was
passed both explicitly and via **call_kwargs. Pop model before building
call_kwargs, and add embed() tests (default + override) — the override
case reproduces the crash.

Also addresses the remaining #1593 review items:
- Drop the redundant global litellm.drop_params mutation; default
  drop_params per-call via setdefault so callers can override it.
- Validate the optional litellm dependency at construction with an
  actionable ImportError.
- Document the litellm provider in the Cloud Providers guide.
- List litellm in the create_client docstring.

Refs #1625
@github-actions github-actions Bot added documentation Documentation changes llm LLM backend changes tests Test changes performance Performance-critical changes labels Jun 12, 2026
@itomek

itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Real-world validation

Verified end-to-end on an AMD Strix Halo (Ryzen AI) box against a live Lemonade backend (OpenAI-compatible), installed via pip install -e ".[litellm]" (litellm 1.88.1) on this branch (f2eb9f3):

  • Live chatcreate_client("litellm").chat(...) routed through litellm → Lemonade and echoed a planted, unguessable token verbatim: CHAT_RESP: 'VIOLET-OTTER-92'
  • embed() model override (the bug) — provider constructed with a chat model, then embed(text, model="…nomic-embed…") — the exact path that raised TypeError: embedding() got multiple values for keyword argument 'model' before this fix — now returns a real 768-dim vector: EMBED_OVERRIDE_OK: True dim: 768
  • Real inference, not a stub — two distinct sentences yield near-orthogonal embeddings (cosine 0.15) ✅

Unit: pytest tests/unit/test_litellm_provider.py12 passed (incl. 2 new embed() tests; the override test reproduces the crash before the fix). Lint: util/lint.py --black --isort --flake8 → all pass.

@itomek itomek marked this pull request as ready for review June 12, 2026 14:20
@itomek itomek requested a review from kovtcharov-amd as a code owner June 12, 2026 14:20
@github-actions

Copy link
Copy Markdown
Contributor

Summary

Clean, focused fix that closes out the remaining #1593 review items. The headline change is a real bug: LiteLLMProvider.embed() passed model both as an explicit arg and inside the spread **call_kwargs, so any caller overriding the model hit TypeError: embedding() got multiple values for keyword argument 'model' — and that path had zero coverage. Popping model from kwargs before building call_kwargs (litellm.py:82) fixes it cleanly, and the new test_embed_uses_override_model reproduces the original crash. The drop_params cleanup (global module mutation → per-call setdefault) is a nice side improvement: it drops a global side-effect on import and lets callers override the default. Docs and the create_client docstring now list the provider. Nothing blocking here.

Issues Found

🟢 Minor — shell-globbing in install hints (docs/sdk/sdks/llm.mdx:10, litellm.py:24)
pip install gaia[litellm] fails under zsh (the default macOS shell), which globs the brackets: zsh: no matches found: gaia[litellm]. Quoting it works in both bash and zsh. Same string appears in the ImportError message — worth fixing both for consistency.

In the docs:

# Requires: pip install 'gaia[litellm]'

And in litellm.py:23-25:

            raise ImportError(
                "litellm is not installed. Install it with: pip install 'gaia[litellm]'"
            ) from e

Strengths

  • Test reproduces the bug before fixing it. test_embed_uses_override_model would have raised the original TypeError; the inline comment naming the failure mode is exactly the kind of context that prevents a future regression.
  • drop_params change is strictly better. Moving from litellm.drop_params = True (global, set on import) to a per-call call_kwargs.setdefault("drop_params", True) removes a hidden global side-effect and keeps caller override intact — matches the no-silent-global-state direction of the codebase.
  • Scope-clean and well-documented. Four files, all directly tied to the stated fix; no drive-by churn. PR description leads with before/after impact and a concrete test plan.

Verdict

Approve with suggestions — the embed fix is correct and tested, and the rest is tidy cleanup. The only items are the optional zsh-quoting nits in the install hints; safe to merge once those are applied (or consciously skipped).

zsh (default macOS shell) globs the brackets in an unquoted
gaia[litellm], failing with 'no matches found'. Quote the install hint
in the docs and the ImportError message so it copy-pastes in bash and
zsh alike.
@itomek

itomek commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator Author

Applied the zsh-quoting suggestion in d6c0e199 — both the docs hint (docs/sdk/sdks/llm.mdx) and the ImportError message now read pip install 'gaia[litellm]', so it copy-pastes cleanly under zsh and bash. Lint + the 12 unit tests still green. Thanks for the catch!

@itomek itomek enabled auto-merge June 12, 2026 14:27
@itomek itomek self-assigned this Jun 12, 2026
The fail-fast 'import litellm' in __init__ was unused, so pylint flagged
W0611 (unused-import); the # noqa: F401 only silenced flake8, not pylint.
Keep a reference (self._litellm) and use it from chat()/embed() instead of
re-importing per call — the import is now genuinely used, no suppressions.
@itomek itomek added this pull request to the merge queue Jun 12, 2026
Merged via the queue into main with commit 93755ba Jun 12, 2026
41 checks passed
@itomek itomek deleted the fix/litellm-embed-model-override branch June 12, 2026 14:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Documentation changes llm LLM backend changes performance Performance-critical changes tests Test changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(llm): LiteLLM embed() crashes on model override + finish #1593 review items

2 participants