NVIDIA Resiliency Extension is a python package for framework developers and users to implement fault-tolerant features. It improves the effective training time by minimizing the downtime due to fa…

Python 3.8+ toolbox for submitting jobs to Slurm

Fast and memory-efficient exact attention

Zero Bubble Pipeline Parallelism

Easily benchmark multiple configurations of LLMs using nanotron.

Meditron is a suite of open-source medical Large Language Models (LLMs).

Reading list of Instruction-tuning. A trend starts from Natrural-Instruction (ACL 2022), FLAN (ICLR 2022) and T0 (ICLR 2022).

distributed trainer for LLMs

An adversarial example library for constructing attacks, building defenses, and benchmarking both