A high-performance Python-based I/O system for large (and small) deep learning problems, with strong support for PyTorch.

ONNX Runtime: cross-platform, high performance ML inferencing and training accelerator

[ICML'24] EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty

llm-export can export llm model to onnx.

TensorRT-LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and build TensorRT engines that contain state-of-the-art optimizations to perform inference efficie…

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

Port of Facebook's LLaMA model in C/C++

Tensor library for machine learning

export llama to onnx

List of Large Lanugage Model Papers

ChatGLM-6B: An Open Bilingual Dialogue Language Model | 开源双语对话语言模型

An experimental open-source attempt to make GPT-4 fully autonomous.

C api for huggingface tokenizer

计图大模型推理库，具有高性能、配置要求低、中文支持好、可移植等特点

INT4 and FP16 inference on CPU for RWKV language model

ChatPiXiu-Survey, reproduction and domain/task adaptation of open source chatgpt alternatives/implementations. PiXiu-貔貅 means fortune.

StableLM: Stability AI Language Models

A repository for single- and multi-modal speaker verification, speaker recognition, and speaker diarization.