💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

An amazing UI for OpenAI's ChatGPT (Website + Windows + MacOS + Linux)

A high-throughput and memory-efficient inference and serving engine for LLMs

A universal scalable machine learning model deployment solution

Fast and memory-efficient exact attention

A framework for few-shot evaluation of language models.

PipeEdge: Pipeline Parallelism for Large-Scale Model Inference on Heterogeneous Edge Devices

Documentation for Colossal-AI

AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration

Port of Facebook's LLaMA model in C/C++

Tensors and Dynamic neural networks in Python with strong GPU acceleration

A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (ggml/gguf), Llama models.

AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

Making large AI models cheaper, faster and more accessible

Training and serving large-scale neural networks

🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.

Enabling PyTorch on Google TPU

🚀 A simple way to train and use PyTorch models with multi-GPU, TPU, mixed-precision

An Open Source Machine Learning Framework for Everyone

End-to-End Object Detection with Transformers

Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch.

Open deep learning compiler stack for cpu, gpu and specialized accelerators