VoXtream is a Full-Stream Zero-shot TTS model with Extremely Low Latency

The official pytorch implemention of the Intespeech 2024 paper "Reshape Dimensions Network for Speaker Recognition"

A method that directly addresses the modality gap by aligning speech token with the corresponding text transcription during the tokenization stage.

Llama-Mimi is a speech language model that uses a unified tokenizer (Mimi) and a single Transformer decoder (Llama) to jointly model sequences of interleaved semantic and acoustic tokens.

jax port of snac

Multi-band Frequency Reconstruction for Neural Psychoacoustic Coding

Codec for paper: LLaSA: Scaling Train-time and Inference-time Compute for LLaMA-based Speech Synthesis

MTLA: Multi-head Temporal Latent Attention

[NAACL 2025] WaveFM: A High-Fidelity and Efficient Vocoder Based on Flow Matching

Awesome speech/audio LLMs, representation learning, and codec models

Official implementation of the paper "BigCodec: Pushing the Limits of Low-Bitrate Neural Speech Codec"

Hackable and optimized Transformers building blocks, supporting a composable construction.

[ICLR 2025] SOTA discrete acoustic codec models with 40/75 tokens per second for audio language modeling

Faster Whisper transcription with CTranslate2

Official code for "F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching"

Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audio codec.

Vector (and Scalar) Quantization, in Pytorch

ICASSP 2024 - Generative De-Quantization for Neural Speech Codec via Latent Diffusion.

Voice Conversion With Just Nearest Neighbors

[ICML 2024] Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model

Mamba SSM architecture

Continual Learning papers list, curated by ContinualAI

Pytorch implementation of Simplified Structured State-Spaces for Sequence Modeling (S5)

Convert Machine Learning Code Between Frameworks

End-to-End Speech Processing Toolkit

Partially Observable Process Gym

An elegant PyTorch deep reinforcement learning library.

An API standard for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)