neural-codec

MLVC: Multi-platform Learned Video Codec for Real-World Deployment

A 1.6B causal Transformer audio tokenizer with streaming, variable bitrates, and semantic alignment across speech, sound, and music

Unofficial PyTorch implementation of Higgs Audio V2 Tokenizer with HuBERT semantic features. Complete training pipeline for semantic-acoustic audio tokenization with 960x downsampling and 8-layer RVQ.

Unofficial PyTorch implementation of VALL-E: zero-shot text-to-speech and voice cloning using neural codec language models. Train and synthesize speech from text with a single reference audio.

Pure-C text-to-speech engine: autoregressive decoder + discrete neural audio codec, no Python at runtime. Sub-realtime on a 2020 M1 CPU (RTF 0.36 int8) and 0.26 on a mainstream NVIDIA GPU. f16/int8/int4 decode weights, memory-mapped, Metal & CUDA optional. Runs NVIDIA Magpie + NanoCodec, 12 languages.

A curated list of the best Text-to-Speech, speech synthesis, and voice-cloning research — models, papers, benchmarks, and toolkits, focused on 2025–2026.

Hide digital data inside speech-shaped audio that survives Zoom, Discord, WhatsApp, and cellular voice. Reproducible Pareto curve of six trained codecs spanning 76 bps (cellular) to 3196 bps (Zoom-class) with listenable demos.

An edge-optimized Vector-Quantized Autoencoder (VQ-VAE) in PyTorch for high-fidelity satellite image compression. Utilizes Sliced Vector Quantization and Gradient Checkpointing to operate within 4GB VRAM, achieving a 19x compression ratio at 0.92 SSIM via a custom composite loss function.

[ACL 2025] OZSpeech: One-step Zero-shot Speech Synthesis with Learned-Prior-Conditioned Flow Matching

Audio-driven facial animation via neural codec features and adaptive channel-grouped losses. Engineered for ultra-fast, production-ready inference. AIアバター向けニューラル音声からアニメーション変換エンジン

Moshi: open-source speech-text foundation model for real-time full-duplex voice dialogue. Uses Mimi neural audio codec. PyTorch, MLX (Apple Silicon) and Rust backends. Moshika & Moshiko voices.

低比特率神经音频编解码器 + 残差矢量量化：把连续音频转成离散 token，服务音频语言模型

Mini SoundStream-style neural audio codec trained on LibriSpeech: 4-codebook RVQ, 3 bitrates, ONNX export

Pure-C port of the RADE V1 and V2 neural HF speech modem

Visual catalog of neural audio and speech codec architectures, audio VAEs and continuous autoencoders, with code, checkpoints, figures and license evidence.

A from-scratch PyTorch implementation of a neural audio codec (Encodec/SoundStream-style) at 3.2 kbps on LibriSpeech, with experiments on perceptual loss for phase recovery in GAN-less settings.