Stars
FlashInfer: Kernel Library for LLM Serving
Achieve state of the art inference performance with modern accelerators on Kubernetes
A high-throughput and memory-efficient inference and serving engine for LLMs
{{ message }}
FlashInfer: Kernel Library for LLM Serving
Achieve state of the art inference performance with modern accelerators on Kubernetes
A high-throughput and memory-efficient inference and serving engine for LLMs