Utsuwa is an open-source alternative to Grok Companion. This is a platform where you can have a virtual AI waifu that learns and grows with you, bundled with optional mechanics inspired by Japanese dating sim games. Utsuwa is privacy-focused - your data lives entirely in your browser.
"Utsuwa" means "vessel" in Japanese - a container for AI to inhabit visually.
- VRM Model Viewer: Load and display VRM 3D avatar models with orbit controls
- Model-Centric UI: Full-screen 3D model with unobtrusive overlay controls
- 3D Speech Bubbles: Chat responses appear as bubbles that track the model's head in 3D space
- Chat Interface: Bottom-centered input bar with streaming responses
- Voice Input: Speech-to-text using Web Speech API with real-time audio visualization
- LLM Integration: Support for 23+ LLM providers including OpenAI, Anthropic, Google, and more
- Text-to-Speech: Support for 13+ TTS providers including ElevenLabs, OpenAI, and Azure
- Lip-sync: Audio-driven mouth animation synced to TTS playback
- Animations: VRMA-based idle and talking animations with automatic blinking
- Character Customization: Customize your companion's name, personality, and system prompt
- Companion System: Multi-axis relationship tracking with mood, events, and semantic memory
- Semantic Memory: Local AI-powered memory search using Transformers.js - finds memories by meaning, not just keywords
- Data Export/Import: Download your data as a save file, restore anytime
- Theming: 4 color themes (Default, Rosé Pine, Tokyo Night, Nord) with light/dark modes
All your data is stored locally in your browser using IndexedDB:
- No database setup required
- Works offline after initial load
- Export/import save files to back up or transfer your data
- Settings > Data to manage your save files
Build a meaningful relationship with your AI companion through a dating sim-inspired progression system:
- Multi-axis Relationships: Track affection, trust, intimacy, comfort, and respect separately
- 8 Relationship Stages: Progress from Stranger → Acquaintance → Friend → Close Friend → Romantic Interest → Dating → Committed → Soulmate
- Dynamic Mood: Real-time emotions with causality tracking (she remembers why she feels a certain way)
- Visual Novel Events: Milestone moments, romantic scenes, and choices that matter - with custom dialogue and branching responses
- Semantic Memory: Facts are indexed with vector embeddings for meaning-based retrieval - "outdoor activities" finds memories about hiking. Runs entirely in-browser using Transformers.js, no API calls
- Natural Progression: Hybrid system combining app heuristics + LLM suggestions for believable relationship growth
- Time-Aware: Your companion notices when you've been away and reacts accordingly
See the Companion System Architecture for full details.
| Category | Providers |
|---|---|
| Major Cloud | OpenAI, Anthropic, Google Gemini, DeepSeek, Mistral, xAI (Grok) |
| Fast Inference | Groq, Cerebras, Fireworks, Together AI |
| Specialized | Perplexity (search), Moonshot (long context) |
| Aggregators | OpenRouter (100+ models), OpenAI Compatible (custom endpoints) |
| Local Servers | Ollama, LM Studio, vLLM |
| Enterprise | Azure AI, Cloudflare Workers AI |
| Additional | Novita, 302.AI, Comet, Player2 |
| Category | Providers |
|---|---|
| Premium Cloud | ElevenLabs, OpenAI TTS, Azure Speech |
| Additional Cloud | Deepgram Aura, Alibaba CosyVoice, VolcEngine, CometAPI |
| Local/Free | Web Speech API (browser), Index-TTS, Browser Local, App Local |
| Generic | OpenAI Compatible TTS, Player2 TTS |
| Provider | Description |
|---|---|
| Web Speech API | Browser-native speech recognition. No API key required. Works in Chrome, Edge, and Safari. |
Voice input is accessed via the microphone button in the chat bar. The audio visualizer provides real-time feedback while speaking.
Note
Utsuwa is in its very early development stages. If you're using the app, save your data often. Early versions may not have backwards-compatible save states and could require manual reformatting.
No installation required! Use Utsuwa directly in your browser at utsuwa.ai
If you prefer to run Utsuwa locally or host your own instance:
- Node.js 22+
- pnpm (recommended) or npm
- A modern browser (Chrome, Firefox, Safari, Edge)
# Clone the repository
git clone https://github.com/dyascj/utsuwa.git
cd utsuwa
# Install dependencies
pnpm install
# Start development server
pnpm devThe app will be available at http://localhost:5173
- Click the Settings (gear icon) in the sidebar
- Navigate to LLM Providers to configure your chat provider:
- Select a provider and enter your API key
- Or use a local server like Ollama or LM Studio
- Navigate to TTS Providers to configure text-to-speech (optional):
- Select a TTS provider
- Enter your API key
- Configure voice settings
All API keys are stored locally in your browser and are never sent to any server except the respective API providers.
- Go to Settings > Avatar
- Click "Load VRM" to select a local
.vrmfile - Or enter a URL to load a VRM model from the web
Your companion data is stored locally in your browser. To back up or transfer your data:
- Go to Settings > Data
- Click Export Save to download a JSON file with all your data
- To restore, click Import Save and select your save file
- Choose Replace (wipe and restore) or Merge (add to existing)
utsuwa/
├── src/
│ ├── lib/
│ │ ├── ai/ # LLM response parsing and prompt building
│ │ ├── assets/ # Static assets
│ │ ├── components/ # Svelte components
│ │ ├── config/ # App and docs configuration
│ │ ├── data/ # Event definitions and static data
│ │ ├── db/ # IndexedDB database (Dexie)
│ │ ├── engine/ # Companion engine (state, memory, events)
│ │ ├── services/ # LLM, TTS, storage services
│ │ ├── stores/ # Svelte 5 stores (state management)
│ │ ├── types/ # TypeScript types
│ │ └── utils/ # Utility functions
│ ├── content/
│ │ └── docs/ # Documentation site content
│ └── routes/
│ ├── (app)/ # Main application routes
│ ├── api/ # API routes
│ └── docs/ # Documentation site routes
├── static/
│ └── models/ # Place default VRM models here
└── package.json
pnpm dev # Start development server
pnpm build # Build for production
pnpm preview # Preview production build
pnpm lint # Type-check and lint the project- VRM model loading and display with orbit controls
- 3D speech bubbles tracking model head position
- Multi-provider LLM support (23+ providers)
- Multi-provider TTS support (13+ providers)
- Audio-driven lip-sync
- VRMA-based animations (idle, talking, blinking)
- Companion system with multi-axis relationships
- 8-stage relationship progression (Stranger → Soulmate)
- Visual novel event system with choices
- Semantic memory system with local embeddings (Transformers.js)
- Time-based mood and relationship decay/recovery
- Local-first IndexedDB storage with export/import
- Theme system with light/dark modes
- Voice input via Web Speech API
- Companion Gender System - Gender selection with male/female specific animations and behaviors
- Multi-provider STT - Support for additional speech-to-text providers beyond Web Speech API
- Enhanced User Controls - More granular control over companion behavior and responses
- Custom Theme Builder - Create and save your own color themes
- Custom Lighting Controls - Adjust 3D scene lighting, environment, and atmosphere
- LLM-Controlled Expressions - Dynamic facial animations and emotions driven by LLM responses
- Live2D Support - Alternative to VRM for 2D animated avatars
- Expanded Default Avatars - More built-in 3D avatar options to choose from
- Photo Mode - Full-featured photo mode with character posing, lighting adjustments, and high-quality saves
- Desktop Application - Native app for Windows, macOS, and Linux with enhanced features
Contributions are welcome! Please read our Contributing Guide for details on how to submit pull requests, report issues, and contribute to the project.
For information about security considerations and how to report vulnerabilities, please see our Security Policy.
Utsuwa is built on the shoulders of these excellent projects:
- Airi - The original inspiration for this project. A beautiful AI companion with VRM avatar support.
- Amica - Open-source AI companion with VRM support and emotional expressions.
- Riko Project by JustRyan - AI waifu project showcasing VRM avatar interactions.
- @pixiv/three-vrm - VRM model loading and rendering for Three.js
- xsAI - Unified LLM and TTS provider integration
- Three.js - 3D graphics engine
- Threlte - Svelte components for Three.js
- SvelteKit - Web application framework
- Tailwind CSS - Utility-first CSS framework
- Transformers.js - In-browser ML for semantic memory embeddings
- bits-ui - Headless UI components for Svelte
- Dexie.js - IndexedDB wrapper for local storage
- simple-icons - SVG icons for provider logos
- n8ao - Ambient occlusion for Three.js
- postprocessing - Post-processing effects
This project is licensed under the MIT License - see the LICENSE file for details.