By eulogik — building AI infrastructure for everyone.
Fugusashi (Japanese: 不縛 — "unbound, unrestrained") is an intelligent model router and multi-agent orchestrator. Named after Fugu Sashi — the famous Japanese pufferfish delicacy — because this router serves up the world's best AI models without the poison of vendor lock-in or the pricing of Sakana Fugu. It automatically picks the best model for each prompt, learns from every request via a CMA-ES coordinator inspired by Sakana's TRINITY paper, and runs entirely on your infrastructure.
Research contributions: (1) Federated routing learning — multiple organizations collaboratively improve routing without sharing data; (2) Human-interpretable routing — every decision comes with a natural language explanation; (3) Continuous CMA-ES adaptation — routing weights evolve from outcomes.
Like Sakana Fugu. But Free. Live Demo · Docs · PyPI · GitHub
Sakana AI's Fugu is a trained orchestration model — a 7B LLM that coordinates frontier models behind a single API. It's powerful, but it's a black box: you can't see why it routes where it does, you can't self-host it, you can't train it on your data, and you pay $5-30 per million tokens plus a $20-200/month subscription.
Fugusashi is the transparent alternative. Same CMA-ES evolution concept (both inspired by Sakana's TRINITY paper), but open, self-hostable, and designed to learn from your specific traffic. Where Fugu hides its routing logic, Fugusashi exposes every decision. Where Fugu runs only in Sakana's cloud, Fugusashi runs on your infrastructure. Where Fugu is static, Fugusashi learns from every request via a feedback loop.
| Feature | Sakana Fugu | Fugusashi |
|---|---|---|
| Architecture | Trained 7B coordinator LLM | CMA-ES evolved weights + rule-based ensemble |
| Orchestration | ✅ TRINITY/Conductor (ICLR 2026) | ✅ Rule-based + GRPO learning |
| Self-Hosting | ❌ Cloud-only | ✅ Local-first, air-gapped |
| Cost | $5-30/M tokens + $20-200/mo | ✅ Free (pay only for model APIs) |
| Transparency | ❌ Black box routing | ✅ Every decision visible + explained |
| Feedback Loop | ❌ Static | ✅ Learns from your traffic |
| Federated Learning | ❌ | ✅ Collaborative routing without data sharing |
| Model Pool | Sakana-controlled frontier models | ✅ You control (100+ providers) |
| Training Data | ❌ Proprietary | ✅ Community preference datasets |
| Dashboard | ❌ | ✅ Live routing visualization |
| EU Availability | ❌ Blocked (GDPR) | ✅ Available everywhere |
| License | Proprietary | ✅ MIT |
Our edge: Fugu is a powerful trained orchestrator with frontier models in its pool. Fugusashi is the transparent, self-hosting alternative you can run on your own infrastructure, train on your own data, and audit every decision. Different bets — Fugu bets on a trained coordinator, we bet on transparency and control.
pip install fugusashidocker run -p 6060:6060 ghcr.io/eulogik/fugusashi:latestgit clone https://github.com/eulogik/fugusashi.git
cd fugusashi
python3 -m venv .venv
source .venv/bin/activate
pip install -e .Edit config.yaml to add your models:
default_model: "llama3.2-local"
models:
- name: "llama3.2-local"
provider: "ollama"
model: "llama3.2:1b"
api_base: "http://localhost:11434"
cost_per_input_token: 0.0
cost_per_output_token: 0.0
capabilities: ["chat", "reasoning"]
description: "Llama 3.2 1B (local, free)"
- name: "gpt-4o-mini"
provider: "openai"
model: "gpt-4o-mini"
cost_per_input_token: 0.00000015
cost_per_output_token: 0.0000006
capabilities: ["chat", "reasoning", "code", "creative"]
description: "OpenAI GPT-4o-mini"fugusashi serve --config config.yaml
# → Fugusashi router listening on 0.0.0.0:6060# Auto-route — the router picks the best model
curl http://localhost:6060/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"auto","messages":[{"role":"user","content":"Say hello"}]}'
# Force a specific model
curl http://localhost:6060/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model":"llama3.2-local","messages":[{"role":"user","content":"Say hello"}]}'Every response includes a routing_decision showing which model was picked, why, and with what confidence.
Three routing strategies in priority order:
- SimilarityRouter — Uses sentence-transformers to find similar past prompts and route to the model that worked best. Gets smarter with every request via the feedback loop.
- CostRouter — Capability-aware routing with cost optimization. Respects
prefer_localfor air-gapped deployments. - FallbackRouter — Always returns a result, even with no data.
A planning model that decomposes hard tasks into subtasks, assigns them to specialist models, and synthesizes results. Uses reinforcement learning (GRPO-style) to learn teamwork patterns.
# Orchestrate a complex task
curl -X POST http://localhost:6060/v1/orchestrate \
-d '{"prompt": "Write a Python web scraper, test it, and write documentation"}'
# View orchestration history
curl http://localhost:6060/v1/orchestration/history
# Check GRPO learning stats
curl http://localhost:6060/v1/orchestration/grpo/statsMultiple Fugusashi instances collaboratively improve a shared routing model without sharing prompts or data. Each organization trains locally, adds differential privacy noise, and contributes weight updates. The result: a router that's smarter than any single deployment.
# Register as a federated client
curl -X POST http://localhost:6060/v1/federated/register \
-d '{"client_id": "my-org", "metadata": {"type": "healthcare"}}'
# Submit local routing updates
curl -X POST http://localhost:6060/v1/federated/submit \
-d '{"client_id": "my-org", "weights": [...], "n_samples": 1000}'
# Trigger aggregation (requires min 3 clients)
curl -X POST http://localhost:6060/v1/federated/aggregateEvery routing decision comes with a natural language explanation:
curl -X POST http://localhost:6060/v1/explain \
-d '{"prompt": "Write a Python class for a binary tree"}'Response:
Decision: Route to gpt-oss-120b (confidence: 87%)
Why: This prompt involves code generation. gpt-oss-120b is best suited for complex reasoning.
Alternatives:
- lfm-2.5-1.2b (12%): better for fast responses
- hermes-3-405b (1%): better for creative writing
Latency: 5.5ms | Strategy: cma-es
Users can override decisions with natural language feedback, which becomes training data.
OpenAI-compatible. Set model: "auto" for intelligent routing.
Response includes routing_decision:
{
"id": "fugu-698f0a66db98",
"model": "llama3.2-local",
"choices": [...],
"routing_decision": {
"model": "llama3.2-local",
"confidence": 0.9,
"strategy": "ensemble(cost)",
"latency_ms": 0.05,
"explanation": "Routed by capability fit + cost"
}
}curl -X POST http://localhost:6060/v1/routing/training \
-H "Content-Type: application/json" \
-d '[{"prompt":"Write Python code","model":"gpt-4o-mini","score":0.95}]'curl -X POST http://localhost:6060/v1/feedback/rate \
-d '{"request_id":"fugu-698f0a66db98","rating":5}'curl -X POST http://localhost:6060/v1/orchestrate \
-d '{"prompt": "Write code, test it, and document it"}'Open http://localhost:6060/dashboard for a live view:
- Overview: total requests, tokens, cost, avg routing latency
- Model Usage: bar chart of which models are being picked
- Strategy Distribution: cost vs similarity routing breakdown
- Recent Decisions: live table of every routing decision
Auto-refreshes every 3 seconds. Light and dark themes.
Fugu's router is static. Fugusashi's learns from every request:
Route → Execute → Evaluate → Learn → (repeat)
- Route: Router picks a model for the prompt
- Execute: Model generates a response
- Evaluate: Track outcome (success/failure, cost, latency)
- Learn: Feed outcomes back into the similarity router
- Auto-Retrain: Every 10 requests, the similarity index rebuilds automatically
Outcomes are stored in .fugusashi_data/outcomes.jsonl — inspectable, shareable, yours.
pip install fugusashi
fugusashi benchmark # Default 20-sample dataset
fugusashi benchmark --train --verbose # With training data
fugusashi benchmark --train --json # JSON output
fugusashi benchmark -d my_data.jsonl # Custom datasetResults:
| Metric | Without Training | With Training |
|---|---|---|
| Accuracy | 70% | 85% |
| Code accuracy | 60% | 90% |
| Strategy | 100% cost | 60% cost / 40% similarity |
| Routing latency | <1ms | ~18ms |
Custom dataset format (JSONL):
{"prompt":"How do I center a div?","expected_model":"gpt-4o-mini","category":"code"}
{"prompt":"What is 2+2?","expected_model":"llama3.2-local","category":"factual"}fugusashi/
├── config.yaml # Model pool + routing config
├── pyproject.toml # Dependencies + metadata
├── LIVING.md # Living development walkthrough
├── README.md # This file
├── LICENSE # MIT
├── src/fugusashi/
│ ├── __init__.py
│ ├── __main__.py # CLI: serve, benchmark, train, expand-data
│ ├── server.py # FastAPI app factory
│ ├── config.py # Pydantic config from YAML
│ ├── providers.py # LiteLLM multi-provider wrapper
│ ├── tracker.py # Cost/routing transparency
│ ├── feedback.py # Feedback loop + learning
│ ├── benchmark.py # Benchmark runner
│ ├── training.py # ModernBERT training pipeline
│ ├── dataset.py # Preference dataset management
│ ├── orchestrator.py # Multi-agent orchestrator
│ ├── grpo.py # GRPO-style reward learning
│ ├── api/
│ │ └── routes.py # All API endpoints
│ ├── router/
│ │ ├── interface.py # Abstract router protocol
│ │ ├── strategies.py # Cost, Similarity, Fallback routers
│ │ ├── learned.py # ModernBERT learned classifier router
│ │ └── ensemble.py # Priority-chain ensemble (learned → similarity → cost)
│ └── static/
│ └── dashboard.html # Live web dashboard
├── tests/
│ ├── test_integration.py # Integration tests
│ ├── test_learned.py # Learned router tests
│ └── test_orchestrator.py # Orchestrator + GRPO tests
└── docs/ # GitHub Pages documentation
- Transparent: Every routing decision is visible and explainable. Fugu is a black box.
- Self-hosting: Runs entirely on-premise with local models via Ollama. Fugu is cloud-only.
- Learning: Gets smarter from your specific traffic via the feedback loop. Fugu is static.
- Federated: Multiple organizations can collaboratively improve routing without sharing data. Fugu doesn't offer this.
- Free: MIT licensed, no usage fees beyond model APIs. Fugu costs $5-30/M tokens + subscription.
- EU-available: No GDPR restrictions. Fugu is blocked in the EU/EEA.
- Yours: Community-owned preference datasets, customizable, extensible. Fugu is Sakana's.
This project is accompanied by a research paper:
Fugusashi: Federated Learning of LLM Routing with Human-Interpretable Decisions
The paper introduces three contributions:
- Federated routing learning — collaborative model routing without data sharing
- Human-interpretable routing — natural language explanations for every decision
- CMA-ES adaptation — continuous evolution of routing weights
| Resource | Link |
|---|---|
| 🌐 Website | eulogik.com |
| 💻 GitHub | github.com/eulogik/fugusashi |
| 📦 PyPI | pypi.org/project/fugusashi |
| 🤗 HF Model | huggingface.co/eulogik/fugusashi-router |
| 📊 HF Dataset | huggingface.co/datasets/eulogik/fugusashi-preferences |
| 🚀 HF Space | huggingface.co/spaces/eulogik/fugusashi |
| 📖 Docs | eulogik.github.io/fugusashi |
| 📝 Paper | paper/main.tex |
| 🌍 eulogik | eulogik.com |
MIT — use it however you want.
Built with ❤️ by eulogik