Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop โ all through one unified, production-ready inference API.
An Easy-to-use, Scalable and High-performance RLHF Framework based on Ray (PPO & GRPO & REINFORCE++ & TIS & vLLM & Ray & Dynamic Sampling & Async Agentic RL)
Structured data extraction and instruction calling with ML, LLM and Vision LLM
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
๐A curated list of Awesome LLM/VLM Inference Papers with Codes: Flash-Attention, Paged-Attention, WINT8/4, Parallelism, etc.๐
GPU cluster manager for optimized AI model deployment
High-performance Inference and Deployment Toolkit for LLMs and VLMs based on PaddlePaddle
Intelligent Router for Mixture-of-Models
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
UltraRAG v2: A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines
Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
ไธไธช่ฝป้็บงใๆฏๆๅ จ้พ่ทฏไธๆไบไบๆฌกๅผๅ็ๅคงๆจกๅๅบ็จ้กน็ฎ(Large Model Data Assistant) ๆฏๆDeepSeek/Qwen3็ญๅคงๆจกๅ ๅบไบ Dify ใLangChain/LangGraphใOllama&VllmใSanic ๅ Text2SQL ๐ ็ญๆๆฏๆๅปบ็ไธ็ซๅผๅคงๆจกๅๅบ็จๅผๅ้กน็ฎ๏ผ้็จ Vue3ใTypeScript ๅ Vite 5 ๆ้ ็ฐไปฃUIใๅฎๆฏๆ้่ฟ ECharts ๐ ๅฎ็ฐๅบไบๅคงๆจกๅ็ๆฐๆฎๅพๅฝขๅ้ฎ็ญ๏ผๅ ทๅคๅค็ CSV ๆไปถ ๐ ่กจๆ ผ้ฎ็ญ็่ฝๅใๅๆถ๏ผ่ฝๆนไพฟๅฏนๆฅ็ฌฌไธๆนๅผๆบ RAG ็ณป็ป ๆฃ็ดข็ณป็ป ๐็ญ๏ผไปฅๆฏๆๅนฟๆณ็้็จ็ฅ่ฏ้ฎ็ญใ
Community maintained hardware plugin for vLLM on Ascend
๐ Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
Evaluate your LLM's response with Prometheus and GPT4 ๐ฏ
Add a description, image, and links to the vllm topic page so that developers can more easily learn about it.
To associate your repository with the vllm topic, visit your repo's landing page and select "manage topics."