The biggest release since 1.0, with 211 commits since v1.0.1. Two flagship systems land here, both still in beta (rough edges expected, feedback very welcome). The default cloud (Gemini Live) path is unchanged, so if you don't opt in, nothing changes for you.

Local Backend (beta)

Gabriel can now run fully offline. Flip the new backend: setting in config.yml to local and the entire pipeline runs on your own hardware.

• Any OpenAI-compatible LLM. LM Studio, Ollama, llama.cpp, vLLM, and similar. Just point local.llm.base_url at your server.
• Moonshine STT plus Silero VAD for fast speech recognition.
• Reuses your existing TTS. Any custom voice provider works (Gemini Live's built-in TTS is the one exception).
• Full feature parity. Tool calling, barge-in and interrupt, reasoning-model support (<think> stripping), and optional vision with a multimodal model.
• Completely optional and zero-footprint. The code is lazy-loaded, and moonshine-voice is an opt-in extra (pip install .[local]).

Spatial Awareness (beta)

Gabriel can now understand the space he's in, not just see it.

• Pose-exfil shader that encodes world position into a color strip (bottom-left) which the Python side screen-captures.
• VRCRaycast sensor rig with 11 rays (forward, sides, diagonals, up and down, ledge, gaze) for walls, ledges, and ceilings.
• Voxel mapping plus pathfinding built live from ray and shader data, routing around obstacles instead of bumping them.
• Raycast-assisted path following that sprints straightaways and dodges doorframes locally instead of stall-replan churn.
• Waypoint system for saving and recalling spots per world.
• One-click Unity installer that auto-wires the VRCFury rig (Full Controller plus armature links).

Plugin System

• Drop-in plugins with tools, TTS, STT, event, chatbox-source, and prompt-contributor hooks.
• ctx.discord namespace extends hooks to the Discord bot's session.
• SafeConfigView, a plugins.trusted toggle, and per-tool toggles via config/tools.yml.
• Bundled plugins moved to the separate ProjectGabriel-Plugins repo, plus new diary (sub-agent journal) and mood (emotion plus intensity) plugins.

Discord and WebUI

• Hybrid Discord message RAG, cross-channel recall, own-message lookup, and KLIPY GIF tools.
• WebUI Vision tab with a live YOLO stream and hot-reloadable tracker tuning.
• Privacy: conversation logging is now opt-in (default off) for both main and Discord.

Under the hood

• Wide refactor of monolithic modules into mixin packages (gemini_live, memory, tts, tracker, mapping_service, voxel_nav, voxel_explorer, wanderer, and control_server into src/web/).
• Pinned google-genai, default model bumped to gemini-3.1-flash-lite, plus numerous tool, prompt, and leak-filter fixes.
• 77 of 77 tests passing.

Upgrade notes

• Default backend stays gemini_live, so no action needed.
• For the local backend: .\bin\uv.exe pip install moonshine-voice==0.0.59 (or pip install .[local]), then point local.llm.base_url at your server.
• Rollback safety: the backup/main-pre-raycast-merge branch exists if the nav stack misbehaves.