Super fast serving stack for LLM on Windows/Linux/Macos

A single-file educational implementation for understanding vLLM's core concepts and running LLM inference.

wheels for TransformerEngine

Achieve state of the art inference performance with modern accelerators on Kubernetes

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

Scalable toolkit for efficient model reinforcement

A simple Python sandbox for helpful LLM data agents

Async RL Training at Scale

🚴 Call stack profiler for Python. Shows you why your code is slow!

Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"

APPS: Automated Programming Progress Standard (NeurIPS 2021)

Realtime log viewer for containers. Supports Docker, Swarm and K8s.

A lightweight data processing framework built on DuckDB and 3FS.

Toolkit for linearizing PDFs for LLM datasets/training

Fast reinforcement learning 💨

A FlashAttention implementation for JAX with support for efficient document mask computation and context parallelism.

Our library for RL environments + evals

A PyTorch native platform for training generative AI models

nanoGRPO is a lightweight implementation of Group Relative Policy Optimization (GRPO)

JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).

FlashMLA: Efficient Multi-head Latent Attention Kernels

Production-tested AI infrastructure tools for efficient AGI development and community-driven innovation

Hardware-Accelerated Reinforcement Learning Algorithms in pure Jax!

[NeurIPS 2025 Spotlight] Reasoning Environments for Reinforcement Learning with Verifiable Rewards

🚀🚀 「大模型」2小时完全从0训练26M的小参数GPT！🌏 Train a 26M-parameter GPT from scratch in just 2h!