Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@thushan
Copy link
Owner

@thushan thushan commented Oct 22, 2025

Documentation updates and various tweaks for v0.0.20.

This version and beyond we flip to using olla as the default engine.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added native support for SGLang, Llamacpp, Lemonade, and Anthropic backends.
    • Updated default proxy engine to Olla for improved performance and features.
  • Documentation

    • Expanded documentation to cover additional supported endpoints and proxy engine options.
    • Added comprehensive Anthropic API integration guide with usage examples and tooling references.
    • Updated project overview to reflect expanded backend compatibility and extensibility.

@coderabbitai
Copy link

coderabbitai bot commented Oct 22, 2025

Walkthrough

Documentation and configuration updates restructuring project overview, backends supported, API endpoints, and development practices. Default proxy engine configuration changed from "sherpa" to "olla". Documentation references updated with reordered badges and extended Anthropic integration details.

Changes

Cohort / File(s) Summary
Core Documentation
CLAUDE.md
Expanded overview of supported backends (SGLang, Llamacpp, Lemonade, Anthropic); added granular commands section; introduced new project structure components including converters, factories, health checks, and translators; restructured proxy implementations; added Hexagonal architecture notes and updated development guidelines.
Configuration
config/config.yaml
Changed default proxy engine from "sherpa" to "olla".
Public Documentation
readme.md,
docs/content/index.md
Reordered and expanded native-support badges (added llama.cpp, LM Deploy, LemonadeSDK); revised Olla description to emphasise broader endpoint compatibility; extended Anthropic Message API section with caution notes, examples, and tooling references; updated native-support/failover paragraphs to mention proxy engines (Sherpa and Olla).
Integration Documentation
docs/content/integrations/api-translation/anthropic.md
Corrected relative paths in Supported Clients links from single-level up (../) to two-level up (../../) references.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes

Possibly related PRs

Suggested labels

documentation

Pre-merge checks and finishing touches

❌ Failed checks (1 inconclusive)
Check name Status Explanation Resolution
Title Check ❓ Inconclusive The title "prepare: v0.0.20" is vague and generic, using a preparation pattern without describing what is actually being changed. Whilst the version number is technically relevant to a release preparation pull request, the title fails to communicate the main substance of the changeset to someone scanning through commit history. The PR objectives clearly indicate that the primary changes include documentation updates and a significant default engine switch from "sherpa" to "olla", neither of which are reflected in the title. The "prepare:" prefix is non-descriptive terminology that doesn't convey meaningful information about the actual modifications. Consider revising the title to better reflect the main changes, such as "docs: update for v0.0.20 and change default engine to olla" or "chore: v0.0.20 release preparation with engine default change". This would give reviewers and future readers immediate clarity about the key modifications in this changeset.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changes. Docstring coverage check skipped.
✨ Finishing touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch prepare/v0.0.20

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f424551 and 39f4e02.

📒 Files selected for processing (5)
  • CLAUDE.md (3 hunks)
  • config/config.yaml (1 hunks)
  • docs/content/index.md (1 hunks)
  • docs/content/integrations/api-translation/anthropic.md (1 hunks)
  • readme.md (3 hunks)
🧰 Additional context used
🪛 LanguageTool
CLAUDE.md

[uncategorized] ~151-~151: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...s, repositories) 4. Security Tests: Rate limiting and size restrictions (see `/test/scrip...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

🪛 markdownlint-cli2 (0.18.1)
CLAUDE.md

8-8: Bare URL used

(MD034, no-bare-urls)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: Cross-Platform Build Validation (windows-latest)
🔇 Additional comments (8)
config/config.yaml (1)

29-32: Default proxy engine correctly updated to "olla" for v0.0.20.

The comment clearly documents the rationale for this change and notes that Sherpa will continue to be maintained. The configuration is properly justified for the release milestone.

docs/content/integrations/api-translation/anthropic.md (1)

43-45: Documentation links correctly updated to reflect directory structure.

The relative paths are now consistently pointing two levels up to the frontend integrations, which aligns with the documentation restructuring. All three links follow the same pattern.

docs/content/index.md (2)

19-22: Badge reorganisation clearly showcases expanded backend support.

The reordering and addition of LM Deploy and LemonadeSDK badges effectively communicates the broader range of natively supported and compatible backends. The new second line separator improves visual organisation.


26-26: Updated descriptions effectively communicate Olla's positioning and flexibility.

The revised description emphasises the "wide variety" of supported endpoints and explicitly mentions the two proxy engine options (Sherpa for simplicity, Olla for performance). This provides users with clear expectations about features like circuit breakers and connection pooling.

Also applies to: 28-28

readme.md (3)

5-5: Badge section consistently updated across project.

The Go version badge and reorganised backend badges match the documentation index updates. The new badges (LM Deploy, LemonadeSDK) and reordered arrangement (with llama.cpp, Ollama, LM Studio) effectively showcase expanded integration coverage.

Also applies to: 9-9, 13-16


35-35: Descriptions align with documentation updates.

The revised language about "wide variety" of natively supported endpoints and explicit mention of proxy engine trade-offs (Sherpa vs Olla with features like circuit breakers) provides clear user guidance.

Also applies to: 37-37


146-159: Anthropic integration section appropriately positioned and cautiously documented.

The new section is well-placed after the OpenWebUI integration and includes a clear caution notice about experimental status (v0.0.20+). The examples reference specific tools (Claude Code, OpenCode, Crush CLI) and provide links to comprehensive integration documentation. The structure matches the existing integration pattern in the README.

CLAUDE.md (1)

4-4: Comprehensive documentation expansion effectively captures new architectural components and testing strategies.

The expanded CLAUDE.md documentation provides excellent detail across multiple dimensions:

  • Backend support (line 4): Clearly lists all supported backends including new Anthropic integration
  • Commands (lines 12-24): Granular make targets improve developer experience and clarity
  • Project structure (lines 28-103): Well-organised hierarchy with new packages (translator, converter, discovery, health, inspector) and reorganised proxy implementations (sherpa/, olla/)
  • API endpoints (lines 108-125): New section clearly documents internal, unified model, proxy, and translator endpoints
  • Response headers (line 141): Extended to include new backends
  • Testing strategy (lines 145-172): Clear categorisation of unit, integration, benchmark, security, and stress testing with specific commands
  • Architecture notes (lines 176-187): New section articulates hexagonal architecture and key components

The documentation aligns well with the v0.0.20 changes and provides clear guidance for developers and users.

Also applies to: 12-24, 28-103, 108-125, 141-141, 145-172, 176-187


The project provides two proxy engines: Sherpa (simple, maintainable) and Olla (high-performance with advanced features).

Full documentation available at: https://thushan.github.io/olla/
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Bare URL should be wrapped for proper markdown formatting.

The static analysis tool flagged this bare URL. For consistency with markdown standards, wrap it in angle brackets or a markdown link format.

Apply this diff to fix the bare URL:

-Full documentation available at: https://thushan.github.io/olla/
+Full documentation available at: <https://thushan.github.io/olla/>
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
Full documentation available at: https://thushan.github.io/olla/
Full documentation available at: <https://thushan.github.io/olla/>
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)

8-8: Bare URL used

(MD034, no-bare-urls)

🤖 Prompt for AI Agents
In CLAUDE.md around line 8, the bare URL "https://thushan.github.io/olla/"
should be wrapped for proper Markdown formatting; replace the bare URL with
either an inline markdown link like
[Documentation](https://thushan.github.io/olla/) or wrap it in angle brackets
<https://thushan.github.io/olla/> so the static analysis warning is resolved and
rendering remains correct.

1. **Unit Tests**: Components in isolation
2. **Integration Tests**: Full request flow through proxy engines
3. **Benchmark Tests**: Performance comparison (balancers, proxy engines, repositories)
4. **Security Tests**: Rate limiting and size restrictions (see `/test/scripts/security/`)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Compound adjective requires hyphenation in Australian English.

The phrase "Rate limiting and size restrictions" should hyphenate the compound adjective when it precedes the noun.

Apply this diff to fix the compound adjective:

-4. **Security Tests**: Rate limiting and size restrictions (see `/test/scripts/security/`)
+4. **Security Tests**: Rate-limiting and size restrictions (see `/test/scripts/security/`)
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
4. **Security Tests**: Rate limiting and size restrictions (see `/test/scripts/security/`)
4. **Security Tests**: Rate-limiting and size restrictions (see `/test/scripts/security/`)
🧰 Tools
🪛 LanguageTool

[uncategorized] ~151-~151: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...s, repositories) 4. Security Tests: Rate limiting and size restrictions (see `/test/scrip...

(EN_COMPOUND_ADJECTIVE_INTERNAL)

🤖 Prompt for AI Agents
In CLAUDE.md around line 151, the compound adjectives need hyphenation; update
the phrase "Rate limiting and size restrictions" to hyphenate the compounds so
it reads "Rate-limiting and size-restrictions" (adjusting both compounds before
the noun) to conform to Australian English hyphenation rules.

@thushan thushan merged commit df959f0 into main Oct 22, 2025
8 checks passed
@thushan thushan deleted the prepare/v0.0.20 branch October 22, 2025 08:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant