See Everything. Control Everything.

Observability, routing, and governance for AI — open source and built for scale.

LLM apps in production often suffer from blind spots: no visibility, unpredictable costs, and fragile vendor lock-in. Brokle solves this as The Open-Source AI Control Plane — See Everything. Control Everything.

• 👁️ See Everything – Track 40+ LLM-specific metrics (latency, token usage, cost, errors, success rates) with complete observability and tracing.
• ⚡ Control Everything – Intelligent routing, failover protection, and governance policies with smart, OpenAI-compatible multi-provider routing.
• 🔒 Stay in Control – Full ownership of your AI stack — open, transparent, and free from lock-in.

👉 Built for production-grade scale, extensibility, and the transparency only open source can provide.

Governance means cost controls, provider policies, and safe-usage guardrails baked right in.

Brokle starts with observability + gateway as the foundation. Next steps include:

• 🔄 Smarter Caching – Semantic-aware caching to slash costs and latency.
• 🚀 Beyond Text – Model hosting and observability for multimodal AI (images, speech, agents).
• 🌐 Complete Control Plane – From observability and routing to caching, multimodal, and compliance — Brokle is becoming the single control plane for all AI workloads.

Step by step, Brokle is evolving into the unified open-source AI control plane — built for the needs of production teams.

• Open Source First – Transparent, extensible, and community-driven.
• Developer-Friendly – Drop-in integrations with OpenAI APIs, LangChain, LlamaIndex, and more.
• Future-Proof – From observability to routing to infrastructure, Brokle grows as your AI stack grows.

# Quick start with Docker
git clone https://github.com/brokle-ai/brokle.git
cd brokle
make setup && make dev

Open http://localhost:3000 to explore Brokle instantly.

Access:
📊 Dashboard: http://localhost:3000
🔌 API: http://localhost:8080
📚 Full guide: docs/DEVELOPMENT.md

• 🚀 Getting Started — Setup and development guide
• 📡 API Reference — REST & WebSocket documentation
• 🏗️ Architecture — System design and technical details
• 🚢 Deployment — Production-ready options
• 🤝 Contributing — Standards and workflow

• Real-time Metrics – Track 40+ LLM-specific indicators (latency, token usage, cost, errors) in real time.
• Quality Scoring – Automated response quality assessment to monitor AI output.
• End-to-End Tracing – Complete visibility into every request and provider interaction.

• Intelligent Provider Selection – ML-powered routing with automatic failover.
• Multi-Provider Smart Routing – Seamless switching between AI providers to prevent downtime.
• Drop-in Compatibility – Works with existing applications, SDKs, and frameworks.

• Built-in Policy Enforcement – Cost controls, usage limits, and safety guardrails built-in.
• Complete Transparency – Full visibility into AI operations with open source flexibility.
• RBAC & Access Control – Enterprise-grade permissions and multi-tenant isolation.
• Real-time Cost Governance – Live spend tracking with intelligent optimization recommendations.

We welcome contributions! See our Contributing Guide to get started.

This repository is MIT licensed, except for the ee/ folders. See LICENSE and docs for more details.

• Website: https://brokle.com
• Documentation: https://docs.brokle.com
• Community: Discord Server
• Twitter: @BrokleAI

Built with ❤️ by the Brokle team. Making AI infrastructure simple and powerful.