Python & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML

Telegram Сaptcha Bot

Anti-spam bot for Telegram and general-purpose anti-spam library and server

Qodo-Cover: An AI-Powered Tool for Automated Test Generation and Code Coverage Enhancement! 💻🤖🧪🐞

APOLLO: SGD-like Memory, AdamW-level Performance; MLSys'25 Oustanding Paper Honorable Mention

Full finetuning of large language models without large memory requirements

Fully open reproduction of DeepSeek-R1

AllenAI's post-training codebase

Open source project for data preparation for GenAI applications

[NeurIPS 2024] BAdam: A Memory Efficient Full Parameter Optimization Method for Large Language Models

Code for Adam-mini: Use Fewer Learning Rates To Gain More https://arxiv.org/abs/2406.16793

Together Mixture-Of-Agents (MoA) – 65.1% on AlpacaEval with OSS models

Collaborative Training of Large Language Models in an Efficient Way

Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients.

GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

Tensors and Dynamic neural networks in Python with strong GPU acceleration

LOMO: LOw-Memory Optimization

Composable building blocks to build LLM Apps

Distily: Language Model Distillation Toolkit and Library

🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

PyTorch native quantization and sparsity for training and inference

g1: Using Llama-3.1 70b on Groq to create o1-like reasoning chains

EvolKit is an innovative framework designed to automatically enhance the complexity of instructions used for fine-tuning Large Language Models (LLMs).

Advanced quantization toolkit for LLMs and VLMs. Support for WOQ, MXFP4, NVFP4, GGUF, Adaptive Schemes and seamless integration with Transformers, vLLM, SGLang, and llm-compressor

Distributed Training Over-The-Internet

Minimalistic large language model 3D-parallelism training

Efficient Triton Kernels for LLM Training