Thanks to visit codestin.com
Credit goes to github.com

Skip to content

RHIDP-15394: add optional id field for inference providers#2106

Open
Jdubrick wants to merge 3 commits into
lightspeed-core:mainfrom
Jdubrick:add-provider-id
Open

RHIDP-15394: add optional id field for inference providers#2106
Jdubrick wants to merge 3 commits into
lightspeed-core:mainfrom
Jdubrick:add-provider-id

Conversation

@Jdubrick

@Jdubrick Jdubrick commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

  • Adds an optional field to set a custom provider_id for inference providers to avoid collision when using multiple of the same provider type

Type of change

  • Refactor
  • New feature
  • Bug fix
  • CVE fix
  • Optimization
  • Documentation Update
  • Configuration Update
  • Bump-up service version
  • Bump-up dependent library
  • Bump-up library or tool used for development (does not change the final image)
  • CI configuration change
  • Konflux configuration change
  • Unit tests improvement
  • Integration tests improvement
  • End to end tests improvement
  • Benchmarks improvement

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

  • Assisted-by: Grok 4.5
  • Generated by: (e.g., tool name and version; N/A if not used)

Related Tickets & Documents

Checklist before requesting a review

  • I have performed a self-review of my code.
  • PR has passed all pre-merge test jobs.
  • If it is a core feature, I have added thorough tests.

Testing

  • Please provide detailed steps to perform tests related to this code change.
  • How were the fix/results from this change verified? Please provide relevant screenshots or results.

Summary by CodeRabbit

  • New Features

    • Added support for optional provider IDs in inference configuration, letting multiple providers of the same type be distinguished clearly.
    • Expanded examples and documentation to show how to configure separate environments with distinct IDs.
  • Bug Fixes

    • Improved validation for provider IDs so invalid values are rejected early.
    • Updated synthesis behavior so explicitly set IDs are preserved, while duplicate IDs are handled consistently.

@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@Jdubrick, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 47 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 395a99c0-4f4e-4e53-a01c-3b42d87e2396

📥 Commits

Reviewing files that changed from the base of the PR and between e93a339 and 1ffcaa8.

📒 Files selected for processing (4)
  • docs/config.puml
  • docs/openapi.json
  • src/client.py
  • src/models/config.py

Walkthrough

Adds an optional id field to UnifiedInferenceProvider with validation (lowercase letters, digits, underscores, hyphens). apply_high_level_inference uses the explicit id when present to synthesize provider_id, falling back to type-derived logic, and logs overwrites. Documentation, schemas, examples, and tests updated accordingly.

Changes

Provider id support

Layer / File(s) Summary
UnifiedInferenceProvider id field and validation
src/models/config.py, tests/unit/models/config/test_inference_configuration.py
Adds an optional id: Optional[str] field with docstring, a field validator stripping whitespace and enforcing allowed characters, and unit tests covering valid/invalid/omitted id cases.
provider_id synthesis logic
src/llama_stack_configuration.py, tests/unit/test_llama_stack_synthesize.py
apply_high_level_inference prefers explicit provider.id for provider_id, else derives from type; logs an info message when overwriting an existing entry; new tests cover explicit id, distinct ids for same type, and duplicate id last-wins behavior.
Schema and documentation updates
docs/config.json, docs/config.html, docs/config.md, docs/config.puml, docs/openapi.json, examples/profiles/inline-faiss.yaml
Generated schemas and docs updated to document the new id field, its constraints, and type literal expansion (adding ollama, vllm); example yaml shows multi-backend usage with distinct ids; unrelated minor description text tweaks in compaction/context-window schemas.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Possibly related PRs

Suggested reviewers: tisnik, radofuchs

🚥 Pre-merge checks | ✅ 6
✅ Passed checks (6 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Performance And Algorithmic Complexity ✅ Passed No blocking perf regressions: the new provider-id handling is a one-time in-memory config pass with no I/O, pagination, or unbounded growth.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main change: adding an optional id field for inference providers.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
✨ Simplify code
  • Create PR with simplified code

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Match generate_openapi_schema.py output (no trailing newline) so spectral
passes, and call setup_logging before library-mode synthesis so INFO
replace logs are visible at boot.

Co-authored-by: Cursor <[email protected]>

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/openapi.json (1)

1-21332: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

CI failure: regenerate the OpenAPI schema.

Pipeline logs report that this file is out of date relative to the generator script.

Run: uv run scripts/generate_openapi_schema.py docs/openapi.json and commit the full regenerated output to ensure it matches source-of-truth exactly (including any unrelated formatting/ordering the generator produces).

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/openapi.json` around lines 1 - 21332, The docs/openapi.json file is
stale and does not match the output of the schema generator, causing CI to fail.
Regenerate this file by running `uv run scripts/generate_openapi_schema.py
docs/openapi.json` from the repository root, then commit the fully regenerated
file as-is (including any formatting or ordering changes produced by the
generator) so it exactly matches the source-of-truth output.

Source: Pipeline failures

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/models/config.py`:
- Line 701: The type annotations in config model use legacy Optional[str] syntax
and should be updated to modern union syntax. Update the field definition for
the id attribute and the related validator signature in the same model to use
str | None instead of Optional[str], keeping the existing behavior unchanged.
Use the config model’s id field and its validator method as the symbols to
locate both spots.

---

Outside diff comments:
In `@docs/openapi.json`:
- Around line 1-21332: The docs/openapi.json file is stale and does not match
the output of the schema generator, causing CI to fail. Regenerate this file by
running `uv run scripts/generate_openapi_schema.py docs/openapi.json` from the
repository root, then commit the fully regenerated file as-is (including any
formatting or ordering changes produced by the generator) so it exactly matches
the source-of-truth output.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 314f34cb-7a62-4038-b1b9-07646386599b

📥 Commits

Reviewing files that changed from the base of the PR and between 93e6dc2 and e93a339.

📒 Files selected for processing (10)
  • docs/config.html
  • docs/config.json
  • docs/config.md
  • docs/config.puml
  • docs/openapi.json
  • examples/profiles/inline-faiss.yaml
  • src/llama_stack_configuration.py
  • src/models/config.py
  • tests/unit/models/config/test_inference_configuration.py
  • tests/unit/test_llama_stack_synthesize.py
📜 Review details
⏰ Context from checks skipped due to timeout. (13)
  • GitHub Check: black
  • GitHub Check: radon
  • GitHub Check: unit_tests (3.12)
  • GitHub Check: unit_tests (3.13)
  • GitHub Check: integration_tests (3.12)
  • GitHub Check: integration_tests (3.13)
  • GitHub Check: build-pr
  • GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-0-7-on-pull-request
  • GitHub Check: E2E: library mode / ci / group 2
  • GitHub Check: E2E: library mode / ci / group 1
  • GitHub Check: E2E: server mode / ci / group 3
  • GitHub Check: E2E: server mode / ci / group 1
  • GitHub Check: E2E Tests for Lightspeed Evaluation job
⚠️ CI failures not shown inline (1)

GitHub Actions: OpenAPI (Spectral) / 0_spectral.txt: RHIDP-15394: add optional id field for inference providers

Conclusion: failure

View job details

##[group]Run set -euo pipefail
 �[36;1mset -euo pipefail�[0m
 �[36;1muv run python scripts/generate_openapi_schema.py /tmp/openapi-generated.json�[0m
 �[36;1mif ! diff -u docs/openapi.json /tmp/openapi-generated.json; then�[0m
 �[36;1m  echo "::error::docs/openapi.json is out of date. Regenerate with: uv run scripts/generate_openapi_schema.py docs/openapi.json"�[0m
🧰 Additional context used
📓 Path-based instructions (3)
tests/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

tests/**/*.py: Use pytest for all unit and integration tests; do not use unittest
Use pytest.mark.asyncio marker for async tests

Files:

  • tests/unit/models/config/test_inference_configuration.py
  • tests/unit/test_llama_stack_synthesize.py
src/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

src/**/*.py: Use absolute imports for internal modules: from authentication import get_auth_dependency
Llama Stack imports: Use from llama_stack_client import AsyncLlamaStackClient
Check constants.py for shared constants before defining new ones
All modules must start with descriptive docstrings explaining purpose
Use logger = get_logger(__name__) from log.py for module logging
All functions must have complete type annotations for parameters and return types, use modern syntax (str | int), and include descriptive docstrings
Use snake_case with descriptive, action-oriented names for functions (get_, validate_, check_)
Avoid in-place parameter modification anti-patterns; return new data structures instead of modifying function parameters
Use async def for I/O operations and external API calls
Use standard log levels with clear purposes: debug() for diagnostic info, info() for program execution, warning() for unexpected events, error() for serious problems
All classes must have descriptive docstrings explaining purpose and use PascalCase with standard suffixes: Configuration, Error/Exception, Resolver, Interface
Abstract classes must use ABC with @abstractmethod decorators
Follow Google Python docstring conventions with required sections: Parameters, Returns, Raises, and Attributes for classes

Files:

  • src/llama_stack_configuration.py
  • src/models/config.py
src/models/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

Pydantic models must use @model_validator and @field_validator for validation and complete type annotations for all attributes, avoiding Any type

Files:

  • src/models/config.py
🧠 Learnings (5)
📚 Learning: 2026-06-24T13:45:37.249Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 1971
File: src/utils/markdown_repair.py:31-36
Timestamp: 2026-06-24T13:45:37.249Z
Learning: In the lightspeed-stack repository, docstrings must use the section header name "Parameters:" (not "Args:") for function arguments, even if the project references Google Python docstring conventions. Ensure docstrings follow the project’s established "Parameters:" header format for any documented function parameters.

Applied to files:

  • tests/unit/models/config/test_inference_configuration.py
  • src/llama_stack_configuration.py
  • src/models/config.py
  • tests/unit/test_llama_stack_synthesize.py
📚 Learning: 2026-05-20T08:09:30.641Z
Learnt from: max-svistunov
Repo: lightspeed-core/lightspeed-stack PR: 1580
File: docs/design/llama-stack-config-merge/poc-results/library-mode/synthesized-run.yaml:107-110
Timestamp: 2026-05-20T08:09:30.641Z
Learning: In Llama-stack config YAMLs, when defining a Llama Guard safety shield entry, set `provider_shield_id` to the *guard model identifier* (e.g., `meta-llama/Llama-Guard-3-8B`). Do not use a chat/generative model id (e.g., `openai/gpt-4o-mini`): a chat-model id (or `native_override`) indicates only an override landed and does **not** mean the safety shield is actually gating queries. Ensure any E2E coverage for the related implementation (JIRA/E2E tests) exercises a real Llama Guard model to verify that the shield is effective.

Applied to files:

  • examples/profiles/inline-faiss.yaml
📚 Learning: 2026-07-06T15:26:18.398Z
Learnt from: Jdubrick
Repo: lightspeed-core/lightspeed-stack PR: 2071
File: src/models/config.py:2416-2422
Timestamp: 2026-07-06T15:26:18.398Z
Learning: In this repo’s Python code under src/**, don’t treat differences in string concatenation style as a style inconsistency when Black has effectively forced (or made clearer) use of explicit `+` string concatenation in multi-line logger/string expressions. If adjacent-literal implicit concatenation is avoided/changed specifically to accommodate Black’s formatting in these call sites, accept the `+` usage and don’t recommend converting it solely for consistency with nearby blocks that use implicit concatenation.

Applied to files:

  • src/llama_stack_configuration.py
  • src/models/config.py
📚 Learning: 2026-01-12T10:58:40.230Z
Learnt from: blublinsky
Repo: lightspeed-core/lightspeed-stack PR: 972
File: src/models/config.py:459-513
Timestamp: 2026-01-12T10:58:40.230Z
Learning: In lightspeed-core/lightspeed-stack, for Python files under src/models, when a user claims a fix is done but the issue persists, verify the current code state before accepting the fix. Steps: review the diff, fetch the latest changes, run relevant tests, reproduce the issue, search the codebase for lingering references to the original problem, confirm the fix is applied and not undone by subsequent commits, and validate with local checks to ensure the issue is resolved.

Applied to files:

  • src/models/config.py
📚 Learning: 2026-02-25T07:46:33.545Z
Learnt from: asimurka
Repo: lightspeed-core/lightspeed-stack PR: 1211
File: src/models/responses.py:8-16
Timestamp: 2026-02-25T07:46:33.545Z
Learning: In the Python codebase, requests.py should use OpenAIResponseInputTool as Tool while responses.py uses OpenAIResponseTool as Tool. This difference is intentional due to differing schemas for input vs output tools in llama-stack-api. Apply this distinction consistently to other models under src/models (e.g., ensure request-related tools use the InputTool variant and response-related tools use the ResponseTool variant). If adding new tools, choose the corresponding InputTool or Tool class based on whether the tool represents input or output, and document the rationale in code comments.

Applied to files:

  • src/models/config.py
🪛 ast-grep (0.44.1)
tests/unit/test_llama_stack_synthesize.py

[warning] 183-183: Do not make http calls without encryption
Context: "http://prod:8000"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)


[warning] 189-189: Do not make http calls without encryption
Context: "http://staging:8000"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)


[warning] 197-197: Do not make http calls without encryption
Context: "http://prod:8000"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)


[warning] 198-198: Do not make http calls without encryption
Context: "http://staging:8000"
Note: [CWE-319] Cleartext Transmission of Sensitive Information.

(requests-http)

🪛 Checkov (3.3.2)
docs/openapi.json

[high] 1-21332: Ensure that the global security field has rules defined

(CKV_OPENAPI_4)


[high] 1-21332: Ensure that security operations is not empty.

(CKV_OPENAPI_5)

🪛 GitHub Actions: OpenAPI (Spectral) / 0_spectral.txt
docs/openapi.json

[error] 1-1: CI check failed: docs/openapi.json is out of date. Regenerate with: uv run scripts/generate_openapi_schema.py docs/openapi.json

🪛 GitHub Actions: OpenAPI (Spectral) / spectral
docs/openapi.json

[error] 1-1: docs/openapi.json is out of date (diff against generated schema failed). Regenerate with: uv run scripts/generate_openapi_schema.py docs/openapi.json

🔇 Additional comments (11)
src/models/config.py (1)

670-673: Validator and docstring look correct.

The validate_id field validator correctly handles None, strips whitespace, rejects empty values, and enforces the [a-z0-9_-]+ character set via re.fullmatch. The docstring uses the "Parameters:" header as required.

Also applies to: 731-761

tests/unit/models/config/test_inference_configuration.py (1)

4-6: LGTM!

Also applies to: 162-193

src/llama_stack_configuration.py (1)

721-728: LGTM!

Also applies to: 748-748, 769-773

tests/unit/test_llama_stack_synthesize.py (1)

131-133: LGTM!

Also applies to: 148-155, 157-173, 175-200, 202-227

examples/profiles/inline-faiss.yaml (1)

14-20: LGTM!

docs/config.html (1)

2108-2112: LGTM!

Also applies to: 2141-2160

docs/config.json (2)

461-461: LGTM!

Also applies to: 578-578, 888-888, 1722-1722


1740-1746: 🗄️ Data Integrity & Integration | ⚡ Quick win

Mirror the id validator in the JSON schema.

id is validated upstream to be trimmed, non-empty, and limited to lowercase letters, digits, underscores, and hyphens, but this schema entry only adds type/nullable/default. If downstream tooling uses this JSON/OpenAPI schema for validation or codegen, invalid ids will still look acceptable until runtime. Please confirm whether the schema generator is expected to surface those rules here; otherwise add the missing machine-readable constraints.

docs/config.md (1)

797-800: LGTM!

Also applies to: 814-814

docs/config.puml (1)

306-307: LGTM!

docs/openapi.json (1)

20358-20369: LGTM!

Also applies to: 20410-20410

Comment thread src/models/config.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant