Fast and memory-efficient exact attention

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

The official Python client for the Hugging Face Hub.

🚀 Efficient implementations of state-of-the-art linear attention models

Development repository for the Triton language and compiler

BitBLAS is a library to support mixed-precision matrix multiplications, especially for quantized LLM deployment.

The best ChatGPT that $100 can buy.

🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

xet client tech, used in huggingface_hub

A high-throughput and memory-efficient inference and serving engine for LLMs

serialize all of Python

A framework for few-shot evaluation of language models.

Pure-Python HTTP/2 protocol implementation

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

Python helpers to limit the number of threads used in native libraries that handle their own internal threadpool (BLAS and OpenMP implementations)

🤗 The largest hub of ready-to-use datasets for ML models with fast, easy-to-use and efficient data manipulation tools

model activation visualiser

Tensors and Dynamic neural networks in Python with strong GPU acceleration

rot13 version of claw code

QQQ is an innovative and hardware-optimized W4A8 quantization solution for LLMs.

Production ready LLM model compression/quantization toolkit with accelerated inference support for both cpu/gpu via HF, vLLM, and SGLang.

SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.

Rigourous evaluation of LLM-synthesized code - NeurIPS 2023 & COLM 2024

5X faster 60% less memory QLoRA finetuning

SOTA Weight-only Quantization Algorithm for LLMs

Official implementation of Half-Quadratic Quantization (HQQ)

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.