Welcome to the llama-cpp-python wiki :)

This wiki provides source-aligned documentation for the public APIs, core classes, feature workflows, examples, and maintainer tooling in llama-cpp-python. The latest code in llama_cpp/ and the corresponding vendored llama.cpp APIs remain the source of truth.

---

Page	Description
Installation	Build and source-installation guide covering Python setup, CMake options, native backends, hardware acceleration, rebuilds, and verification.

Page	Description
Llama	Main high-level interface for GGUF loading, text and chat completion, tokenization, embeddings, sampling, speculative decoding, caching, and lifecycle management.

Page	Description
Llama Cache	Cache interfaces and implementations for reusing model state across repeated prompts.
Llama Embedding	Dedicated embedding APIs, configuration, output formats, and batching behavior.
Llama Grammar	Grammar parsing and constrained-generation utilities.
Llama Speculative Decoding	Stateful MTP, DFlash, DFlash2, DSpark, and n-gram engines; configuration, lifecycle, rollback, statistics, and benchmarks.
Logger	Python and native logging configuration, callbacks, levels, filtering, and output routing.
llama.cpp ctypes Bindings	Source-oriented reference for the low-level llama.cpp and ggml ctypes bindings.
MTMD ctypes Bindings	Source-oriented reference for the low-level multimodal ctypes bindings.

Page	Description
Embeddings and Reranking	End-to-end sentence embeddings, token-level vectors, normalization, streaming batches, similarity output, and cross-encoder reranking.

Page	Description
DFlash2 Speculative Decoding	Configure a DFlash2 sidecar, validate selector execution, compare ordinary and speculative output, and tune draft length.
MTMD Speech Synthesis	Generate speech with Qwen3-TTS Base or Pocket TTS; configure reference audio and FA, and explore CLI and Streamlit examples.

Page	Description
Git Commit Generation Agent	Maintainer workflow for producing clear, structured, and source-aware Git commit messages.

Page	Description
Wiki Schema	Documentation structure, page templates, source requirements, and maintenance rules.
Contributing to the Wiki	Contribution workflow for creating and updating documentation.

---

For general model loading and generation:

1. Installation
2. Llama
3. Llama Cache
4. Llama Grammar
5. Logger

For embeddings and reranking:

1. Llama Embedding
2. Embeddings and Reranking

For speculative decoding:

1. Llama
2. Llama Speculative Decoding
3. DFlash2 Speculative Decoding

For text-to-speech:

1. MTMD Speech Synthesis — Qwen3-TTS Base and Pocket TTS, reference audio, Flash Attention, output formats, and current cloning limitations.
2. CLI TTS Example — single requests, multilingual scenarios, and batch synthesis.
3. Streamlit TTS Playground — reference upload/recording, playback, and downloads.

For documentation contributors:

1. Wiki Schema
2. Contributing to the Wiki
3. Git Commit Generation Agent

---

Completed pages currently linked from this index:

• install.md
• core/Llama.md
• modules/LlamaCache.md
• modules/LlamaEmbedding.md
• modules/LlamaGrammar.md
• modules/LlamaSpeculative.md
• modules/Logger.md
• modules/LlamaCppBindings.md
• modules/MTMDCppBindings.md
• features/embeddings-rerank.md
• examples/dflash2-speculative-decoding.md
• examples/audio/audio-tts.md
• development/git-commit-generation-agent.md
• SCHEMA.md
• contributing-to-wiki.md

The repository also contains empty placeholder files for planned documentation. They are intentionally not linked as usable pages until content has been added and checked against the implementation.

• Basic and chat-completion examples
• Vision and audio-input examples
• Caching, grammar, multi-model, and tool-call feature guides
• Common and MCP type references
• Troubleshooting and backend diagnostics

---

• Treat source code as the source of truth.
• Keep parameters, defaults, version availability, and behavior aligned with the latest implementation.
• Prefer complete, runnable examples without local machine-specific paths.
• Clearly mark deprecated APIs, preview features, and current limitations.
• Distinguish stable public interfaces from private implementation helpers.
• Do not link empty placeholder pages as finished documentation.
• Keep pages concise, practical, and easy to navigate.

---

• llama-cpp-python on GitHub
• Installation guide
• Wiki schema
• Contribution guide