Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure. Learn more →
Top 23 Python llmops Projects
-
Project mention: DeepSeek-OCR: When a Picture Is Actually Worth 10 Fewer Tokens | dev.to | 2025-10-26
One gotcha: if you're using vLLM, you'll need the 0.8.5 wheel for CUDA 11.8. Download it from vLLM releases before installing.
-
InfluxDB
InfluxDB – Built for High-Performance Time Series Workloads. InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
-
litellm
Call any LLM API with cost tracking, guardrails, logging and load balancing. 1.8k+ models, 80+ providers, 50+ endpoints (unified + native format). Available as a Python SDK or Proxy Server (AI Gateway).
[BerriAI/litellm]: LiteLLM - A simple library to call any LLM API
-
-
SuperAGI
<⚡️> SuperAGI - A dev-first open source autonomous AI agent framework. Enabling developers to build, manage & run useful autonomous agents quickly and reliably.
-
opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
For monitoring, there are separate full-fledged monitoring solutions like Opik, PostHog, Langfuse or OpenLLMetry, maybe will try some next time.
-
OpenLLM
Run any open-source LLMs, such as DeepSeek and Llama, as OpenAI compatible API endpoint in the cloud.
Project mention: Your 2025 Roadmap to Becoming an AI Engineer for Free for Vue.js Developers | dev.to | 2025-08-06REST APIs to connect AI models to Vue.js apps (example 1, example 2).
-
Project mention: Evalúa y Mejora Tus Agentes: Evaluación Automatizada con RAGAS para Agentes de Producción | dev.to | 2025-10-15
-
Stream
Stream - Scalable APIs for Chat, Feeds, Moderation, & Video. Stream helps developers build engaging apps that scale to millions with performant and flexible Chat, Feeds, Moderation, and Video APIs and SDKs powered by a global edge network and enterprise-grade infrastructure.
-
Project mention: Metaflow: Build, Manage and Deploy AI/ML Systems | news.ycombinator.com | 2025-07-16
Stay tuned! We have some cool new features coming soon to support agentic workloads (teaser: https://github.com/Netflix/metaflow/pull/2473)
If you are curious, join the Metaflow Slack at http://slack.outerbounds.co and start a thread on #ask-metaflow
-
BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Project mention: My personal favorite MCP server which has became part of my life | dev.to | 2025-05-27 -
For monitoring, there are separate full-fledged monitoring solutions like Opik, PostHog, Langfuse or OpenLLMetry, maybe will try some next time.
-
-
Seamless integration: Works with OCI-compliant registries (e.g., Docker Hub and Jozu Hub) and integrates with popular tools like HuggingFace, ZenML, and Git.
-
Giskard is like unit testing but for AI models. It helps you identify and fix issues like bias, hallucinations, or incorrect outputs before your AI reaches users. This tool is essential for quality control in production AI applications.
-
cognita
RAG (Retrieval Augmented Generation) Framework for building modular, open source applications for production by TrueFoundry
Project mention: Lists of open-source frameworks for building RAG applications | dev.to | 2025-01-02Ideal For: Enterprises seeking a robust framework for large-scale AI applications. GitHub Repository
-
-
AGiXT
AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.
-
[TruLens Guide]: Web Search Agent Evaluation with TruLens
-
agent-starter-pack
A collection of production-ready Generative AI Agent templates built for Google Cloud. It accelerates development by providing a holistic, production-ready solution, addressing common challenges (Deployment & Operations, Evaluation, Customization, Observability) in building and deploying GenAI agents.
-
uptrain
UpTrain is an open-source unified platform to evaluate and improve Generative AI applications. We provide grades for 20+ preconfigured checks (covering language, code, embedding use-cases), perform root cause analysis on failure cases and give insights on how to resolve them.
Project mention: Launch HN: Lucidic (YC W25) – Debug, test, and evaluate AI agents in production | news.ycombinator.com | 2025-07-30 -
-
LLMStack
No-code multi-agent framework to build LLM Agents, workflows and applications with your data
Project mention: Show HN: Langrocks – tools like computer access, browser etc., for LLM agents | news.ycombinator.com | 2024-11-20We built tools like web browser, code interpreter etc., needed for LLM agents as part of our LLMStack project. We have now moved them into a single collection as langrocks. We're using this in LLMStack and thought others might find it useful. https://github.com/trypromptly/LLMStack/blob/main/llmstack/p... shows how we use langrocks with Anthropic's Claude with computer use to automate web browsing.
The coolest part is watching an LLM actually use a computer - you get a unique URL to view the virtual display, so you can see exactly what it's doing with tools like computer access and web browser. We've used this to automate complex workflows where the LLM needs to research across multiple sites, interact with web apps, or perform system operations.
-
openlit
Open source platform for AI Engineering: OpenTelemetry-native LLM Observability, GPU Monitoring, Guardrails, Evaluations, Prompt Management, Vault, Playground. 🚀💻 Integrates with 50+ LLM Providers, VectorDBs, Agent Frameworks and GPUs.
OpenLIT is an OpenTelemetry-native observability tool built for AI Engineering and LLM applications. It focuses on easy, vendor-neutral instrumentation for LLMs, vector databases, and other AI stack components. OpenLIT is ideal for teams already heavily invested in OpenTelemetry and GPU monitoring but offers fewer features for LLM prompt evaluation and experimentation.
-
burr
Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.
Project mention: Diátaxis – A systematic approach to technical documentation authoring | news.ycombinator.com | 2024-12-04- https://burr.dagworks.io
It's not always the easiest to follow (we often have disagreements about whether something is a tutorial or a how-to), but it's a really valuable framing and I think our docs have gotten better because of it.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python llmops discussion
Python llmops related posts
-
DeepSeek-OCR: When a Picture Is Actually Worth 10 Fewer Tokens
-
Evalúa y Mejora Tus Agentes: Evaluación Automatizada con RAGAS para Agentes de Producción
-
[KubeRay로 LLM 서빙 인프라 찍먹] 3부: vLLM과 Ray Serve를 활용한 고성능 추론 엔드포인트 구축기
-
LLM Observability in the Wild – Why OpenTelemetry Should Be the Standard
-
5 Practical AI Stacks for Anyone Not Named Google
-
Qwen3-Next Complete Technical Analysis: Major Breakthrough in AI Model Architecture for 2025
-
Building Strands Agents with a few lines of code: Evaluating Performance with RAGAs
-
A note from our sponsor - Stream
getstream.io | 16 Nov 2025
Index
What are some of the best open-source llmops projects in Python? This list will help you:
| # | Project | Stars |
|---|---|---|
| 1 | vllm | 62,592 |
| 2 | litellm | 31,037 |
| 3 | serve | 21,784 |
| 4 | SuperAGI | 16,853 |
| 5 | opik | 15,630 |
| 6 | OpenLLM | 11,928 |
| 7 | ragas | 11,383 |
| 8 | metaflow | 9,621 |
| 9 | BentoML | 8,193 |
| 10 | openllmetry | 6,582 |
| 11 | superduper | 5,228 |
| 12 | zenml | 5,006 |
| 13 | giskard-oss | 4,972 |
| 14 | cognita | 4,277 |
| 15 | lorax | 3,532 |
| 16 | AGiXT | 3,123 |
| 17 | trulens | 2,914 |
| 18 | agent-starter-pack | 2,857 |
| 19 | uptrain | 2,326 |
| 20 | llm-guard | 2,228 |
| 21 | LLMStack | 2,064 |
| 22 | openlit | 2,013 |
| 23 | burr | 1,834 |
Sponsored
InfluxDB – Built for High-Performance Time Series Workloads
InfluxDB 3 OSS is now GA. Transform, enrich, and act on time series data directly in the database. Automate critical tasks and eliminate the need to move data externally. Download now.
www.influxdata.com