Pinned Loading

1. sgl-project/sglang sgl-project/sglang Public

   SGLang is a high-performance serving framework for large language models and multimodal models.

   Python 22.1k 3.9k

2. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 66.7k 12.3k

3. LightLLM LightLLM Public

   Forked from ModelTC/LightLLM

   LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.

   Python

4. lmdeploy lmdeploy Public

   Forked from InternLM/lmdeploy

   LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

   Python

5. TensorRT-LLM TensorRT-LLM Public

   Forked from NVIDIA/TensorRT-LLM

   TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

   C++

6. 3D_cardiac_multi-structure_segmentation 3D_cardiac_multi-structure_segmentation Public

   Python