A local-first browser extension for chat with local LLM providers (Ollama, LM Studio, llama.cpp) from the browser sidepanel.

Quick links: Install · Docs · Setup Guide · Privacy · Issues

Ollama Client is a sidepanel chat extension for Chromium browsers, with Firefox support.

It is a local LLM Chrome extension where you choose provider endpoints and models.

Version v0.6.0 introduced multi-provider chat routing and local RAG workflows.

• Developers who want an Ollama client directly in the browser UI.
• Users who want an offline AI assistant workflow with local storage by default.
• Users running local provider servers (Ollama, LM Studio, llama.cpp).
• Contributors interested in browser extension + local model architecture.

• Users expecting cloud-SaaS reliability without running local infrastructure.
• Teams requiring centralized cloud sync, SSO, and org admin controls.
• Users who do not want to manage provider endpoints.

Most AI browser tools assume hosted providers and account-based workflows.

This project focuses on local-first usage:

• You configure the model endpoint.
• Chat/session data is stored locally.
• There is no built-in telemetry pipeline.

• Sidepanel-native browser UX instead of a separate desktop app.
• Multi-provider routing in one UI.
• Built-in local retrieval flow (RAG with local LLMs).
• Source-visible behavior for auditing and contribution.

High-level flow:

1. Sidepanel/options UI collects prompt and settings.
2. UI opens runtime port to background.
3. Background resolves provider by selected model mapping.
4. Provider stream is relayed back to UI in chunks.
5. UI updates message state and persists chat data.
6. Optional RAG pipeline retrieves local context and appends it to prompt input.

Key directories:

• src/sidepanel/*
• src/options/*
• src/background/*
• src/contents/*
• src/lib/providers/*
• src/lib/embeddings/*

Build/runtime notes:

• Extension framework: WXT (wxt CLI)

  moved from Plasmo to WXT for more deterministic MV3 builds and explicit entrypoint/manifest control.

• Settings hooks/storage wrapper: @plasmohq/storage (plasmoGlobalStorage)

Default provider profiles:

• Ollama (http://localhost:11434)
• LM Studio (http://localhost:1234/v1)
• llama.cpp server (http://localhost:8000/v1)

Clarifying examples:

• If both Ollama and LM Studio expose llama3, mapping decides which backend handles chat.
• If mapping is absent for a model ID, fallback provider is Ollama.

In this project, RAG means local retrieval before generation:

1. Uploaded or chat text is chunked.
2. Chunks are embedded and stored in local vector storage.
3. Query-time retrieval selects relevant chunks.
4. Retrieved snippets are appended to generation context.

Clarifying example:

• Upload a local API spec PDF, then ask: What headers are required for createUser?
• Retrieved chunks from that PDF are included in prompt context before model response.

Current RAG runtime is intentionally browser-first:

• extension context only (UI + background worker)
• IndexedDB + in-memory index/cache
• HTTP-based model/embedding access
• graceful fallback over hard failure

Embedding strategy defaults:

• provider-native embeddings when available
• shared canonical target: all-MiniLM-L6-v2
• silent background warmup
• Ollama fallback for reliability

RAG implementation details and module boundaries:

• Current behavior guide: docs/rag.md
• Full audit and redesign: docs/rag-browser-core.md
• Browser-first RAG contracts (TypeScript interfaces): src/lib/rag/core/interfaces.ts

1. Install extension from the Chrome Web Store.
2. Start at least one provider endpoint.
3. Configure provider URL in settings.
4. Select a model and start chatting in the sidepanel.

Common endpoint examples:

• http://localhost:11434 (Ollama)
• http://localhost:1234/v1 (LM Studio)
• http://localhost:8000/v1 (llama.cpp server)

git clone https://github.com/Shishir435/ollama-client.git
cd ollama-client
pnpm install
pnpm dev

Common commands:

pnpm lint:check
pnpm test:run
pnpm build
pnpm package

Firefox commands:

pnpm dev:firefox
pnpm build:firefox
pnpm package:firefox

1. Start provider service.
2. Open extension settings and verify connection.
3. Select model.
4. Send prompt and monitor stream.
5. Optionally upload files for retrieval context.
6. Fork conversation by editing earlier user messages.

• Enable only providers you need.
• Keep model names unique when possible.
• Re-check mappings after model list changes.

• Per-model parameters are stored locally (temperature, top_p, top_k, etc.).
• Start with defaults, then tune one variable at a time.

• Adjust chunk size/overlap and retrieval thresholds.
• Narrow retrieval scope when debugging noisy answers.

• Legacy key/message naming (ollama-*) remains in parts of multi-provider code.
• Embedding support varies by provider; fallback keeps Ollama as reliability anchor.
• Reranker exists but is disabled by default due extension CSP constraints.
• Provider parity is incomplete for model management actions.
• Runtime persistence is Dexie-first while SQLite migration path still exists.

• Privacy depends on endpoint choice.
• If you configure a remote endpoint, prompts/responses are sent to that endpoint.
• Do not expose provider APIs publicly without access controls.

1. Provider-agnostic naming cleanup.
2. Clear single-source persistence strategy.
3. Better provider parity for management actions.
4. Better retrieval diagnostics.

Potential future architecture may include a desktop helper/local companion for heavier retrieval workloads.

Important constraints:

• this is not implemented
• browser-only mode remains first-class
• core runtime does not depend on helper availability

• Read CONTRIBUTING.md.
• Keep PRs scoped and testable.
• Include reproduction details for bug fixes.
• Update docs when behavior changes.

Philosophy:

• Local-first operation.
• Explicit behavior over hidden automation.
• User control of endpoint and model settings.

Non-goals:

• Managed cloud LLM platform behavior.
• Hidden telemetry for growth metrics.
• Abstracting away all local infrastructure responsibility.

MIT License: LICENCE

• Architecture Guide
• Provider Support
• RAG Guide
• Browser-First RAG Core (Audit + Design)
• Contributing Guide
• Setup Guide (Web)
• Privacy Policy (Web)