Awesome TTS & Voice Generation Models

A curated list of open-source Text-to-Speech (TTS), voice cloning, and music generation models. Models are sorted by release date (newest first).

---

Table of Contents

• Text-to-Speech (TTS) Models
• Music Generation Models
• Anything to Audio
• Audio Restoration & Enhancement
• Speech Recognition (ASR)
• Additional Resources

---

Text-to-Speech (TTS) Models

TTS Quick Comparison

Model	Voice Cloning	ASR	Languages	Streaming	License
LongCat-AudioDiT	✅	❌	Zh/En	❌	MIT
VoxCPM2	✅	❌	30	✅	Apache-2.0
MOSS-TTS-Nano	✅	❌	20	✅	Apache-2.0
T5Gemma-TTS	✅	❌	En/Zh/Jp	❌	MIT
TinyTTS	❌	❌	En	✅	Apache-2.0
LEMAS-TTS	✅	❌	10	❌	Apache-2.0
OmniVoice	✅	❌	600+	❌	Apache-2.0
LongCat-Next	✅	✅	Zh/En	✅	MIT
Voxtral-4B-TTS	✅	❌	9	✅	CC BY-NC 4.0
Irodori-TTS-500M-v2	✅	❌	Jp	❌	MIT
Fish Audio S2 Pro	✅	❌	80+	✅	Research License
KittenTTS	✅	❌	En+	✅	Apache-2.0
MOSS-TTS	✅	❌	20	✅	Apache-2.0
SoulX-Singer	✅ (Singing)	❌	Zh/En/Canto	✅	Apache-2.0
SoproTTS	✅	❌	En	✅	Apache-2.0
NeuTTS	✅	❌	En/Es/De/Fr	✅	Apache-2.0
Qwen3-TTS	✅	❌	10	✅	Apache-2.0
GLM-TTS	✅	❌	Zh/En	✅	Apache-2.0
VibeVoice-Realtime	✅	❌	Multi	✅	MIT
Fun-CosyVoice 3.0	✅	❌	9 + 18 dialects	✅	Apache-2.0
MioTTS-2.6B	✅	❌	En/Jp	✅	LFM
Supertonic 2	❌	❌	5	✅	OpenRAIL-M
KugelAudio	✅	❌	23 EU	✅	MIT
Kokoro-82M	✅	❌	8 (54 voices)	✅	Apache-2.0
KokoClone	✅	❌	7	✅	Apache-2.0
IndexTTS2	✅	❌	Zh/En	✅	Apache-2.0
Maya1	✅	❌	En	✅	Apache-2.0
LFM2-Audio-1.5B	✅	✅	En	✅	LFM
Step-Audio-EditX	✅	❌	Zh/En/Jp/Ko	✅	Apache-2.0
FireRedTTS2	✅	❌	7 langs	✅	Apache-2.0
VoxCPM	✅	❌	Zh/En	✅	Apache-2.0
LuxTTS	✅	❌	-	✅	Apache-2.0
MegaTTS3	✅	❌	Zh/En	✅	Apache-2.0
Spark-TTS	✅	❌	Zh/En	✅	Apache-2.0
Fish Speech	✅	❌	8 langs	✅	Apache-2.0
Step-Audio	✅	✅	Zh/En/Jp	✅	Apache-2.0
SoulX-Podcast	✅	❌	Zh/En/Canto	✅	Apache-2.0
Chatterbox	✅	❌	23+	✅	MIT
Orpheus-TTS	✅	❌	Multi	✅	Apache-2.0
Dia	✅	❌	En	✅	Apache-2.0
VieNeu-TTS	✅	❌	Vi	✅	Apache-2.0
MiMo-Audio	✅	✅	Multi	✅	Apache-2.0
Kimi-Audio	✅	✅	Multi	✅	MIT/Apache-2.0
ZipVoice	✅	❌	Zh/En	✅	Apache-2.0

LongCat-AudioDiT

LongCat-AudioDiT

Description: State-of-the-art diffusion-based TTS model operating directly in waveform latent space. Developed by Meituan's LongCat team, it requires only a Waveform VAE and Diffusion backbone, effectively mitigating compounding errors.

Release Date: March 30, 2026

Feature	Value
Parameters	1B / 3.5B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	❌
Languages	Chinese, English
Streaming	❌
Sample Rate	24000 Hz
License	MIT

Key Innovation: Adaptive Projection Guidance (APG) replaces traditional classifier-free guidance for elevated generation quality. Outperforms Seed-TTS on zero-shot voice cloning benchmarks.

Links:

VoxCPM2

VoxCPM2

Description: OpenBMB's next-generation tokenizer-free diffusion autoregressive TTS model with 2 billion parameters. Supports 30 languages with automatic detection, voice design from text descriptions, and high-fidelity voice cloning.

Release Date: 2026

Feature	Value
Parameters	2B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅ (Voice Design)
Languages	30 (+ 9 Chinese dialects)
Streaming	✅ (RTF ~0.3)
Audio Output	48 kHz
License	Apache-2.0

Key Innovation: Tokenizer-free design with LocEnc → TSLM → RALM → LocDiT pipeline. Built-in super-resolution via AudioVAE V2 for 48kHz output.

Links:

MOSS-TTS-Nano

MOSS-TTS-Nano

Description: Ultra-lightweight open-source multilingual speech generation model with only 0.1B parameters. Designed for realtime speech generation that runs directly on CPU without GPU.

Release Date: 2026

Feature	Value
Parameters	0.1B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	❌
Languages	20
Streaming	✅ (CPU-friendly)
Audio Output	48 kHz Stereo
License	Apache-2.0

Key Innovation: Pure autoregressive architecture with MOSS-Audio-Tokenizer-Nano. Compresses audio to 12.5 Hz token stream using RVQ with 16 codebooks. Runs on 4-core CPU.

Links:

T5Gemma-TTS

T5Gemma-TTS

Description: Multilingual TTS model with voice cloning and duration control, built on the T5Gemma encoder-decoder LLM architecture. Supports batch generation for multiple audio variations.

Release Date: 2026

Feature	Value
Parameters	2B-2B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	❌
Languages	English, Chinese, Japanese
Streaming	❌
VRAM	7.6-10.6 GB
License	MIT

Key Innovation: PM-RoPE positional encoding with XCodec2 audio codec. Low-VRAM options with CPU offloading. Batch inference efficiency with single encoder pass.

Links:

TinyTTS

TinyTTS

Description: The smallest English TTS model with only 1.6 million parameters. End-to-end neural network achieving ~53x real-time synthesis speed on CPU via ONNX optimization.

Release Date: 2026

Feature	Value
Parameters	1.6M
Zero-shot Voice Cloning	❌
ASR	❌
Pronunciation Control	✅
Emotion Control	❌
Languages	English
Streaming	✅ (~53x RTF)
Model Size	~3.4 MB (ONNX FP16)
License	Apache-2.0

Key Innovation: Ultra-compact architecture optimized for CPU-only deployment. Multi-platform support via Python and Node.js APIs. Works on laptops, edge devices, and embedded systems.

Links:

LEMAS-TTS

LEMAS-TTS

Description: Part of the LEMAS (Large-scale Extensible Multilingual Audio Suite) project. Zero-shot multilingual TTS with 0.3B parameters supporting 10 languages with word-level precise editing capabilities.

Release Date: 2026

Feature	Value
Parameters	0.3B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	10 (zh/en/de/fr/es/pt/it/ru/id/vi)
Streaming	❌
Special Feature	Word-level editing (LEMAS-Edit)
License	Apache-2.0

Key Innovation: Built on 150,000+ hours of multilingual speech data with word-level timestamps. Includes LEMAS-Edit for precise word-level speech editing via masked token infilling.

Links:

OmniVoice

OmniVoice

Description: Massive multilingual zero-shot TTS model scaling to 600+ languages. Uses diffusion language model-style discrete non-autoregressive architecture with single-stage text-to-acoustic mapping.

Release Date: 2026

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅ (Pinyin/CMU)
Emotion Control	✅ (Voice Design)
Languages	600+
Streaming	❌
Training Data	581k hours
License	Apache-2.0

Key Innovation: Simplified single-stage architecture vs conventional two-stage pipelines. Full-codebook random masking strategy with LLM initialization for superior intelligibility. Noise-robust prompt processing.

Links:

LongCat-Next

LongCat-Next

Description: Native multimodal foundation model by Meituan LongCat Team processing text, vision, and audio under a single autoregressive objective. Industrial-strength model with strong speech synthesis and voice cloning.

Release Date: March 2026

Feature	Value
Parameters	3B (MoE A3B)
Zero-shot Voice Cloning	✅
ASR	✅
Pronunciation Control	✅
Emotion Control	✅
Languages	Chinese, English
Streaming	✅ (Low latency)
Audio Output	24 kHz
License	MIT

Key Innovation: Discrete Native Autoregression Paradigm (DiNA) unifying modalities in shared discrete token space. Combines visual understanding, generation, and audio processing in single model.

Links:

Voxtral-4B-TTS

Voxtral-4B-TTS

Description: Frontier, open-weights text-to-speech model developed by Mistral AI. Designed to be fast, instantly adaptable, and produces lifelike speech with natural prosody and emotional range.

Release Date: March 2026

Feature	Value
Parameters	4B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	✅ (expressive speech)
Languages	9 (En, Fr, Es, De, It, Pt, Nl, Ar, Hi)
Streaming	✅ (RTF 0.103 at concurrency 1)
Audio Output	24 kHz
License	CC BY-NC 4.0

Links:

Irodori-TTS-500M-v2

Irodori-TTS-500M-v2

Description: Japanese Text-to-Speech model based on Rectified Flow Diffusion Transformer. Features emoji-based style and sound effect control by embedding emojis in input text for expressive speech generation.

Release Date: 2026

Feature	Value
Parameters	500M
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	✅ (emoji-based)
Languages	Japanese
Streaming	❌
Output Quality	48kHz waveform
License	MIT

Key Feature: Emoji annotation control - insert specific emojis into text to control speaking styles, emotions, and sound effects.

Links:

Fish Audio S2 Pro

Fish Audio S2 Pro

Description: Fish Audio S2 Pro is a leading text-to-speech model with fine-grained inline control of prosody and emotion. It combines reinforcement learning alignment with a dual-autoregressive architecture for high-quality speech synthesis.

Release Date: March 10, 2026

Feature	Value
Parameters	5B (4B Slow AR + 400M Fast AR)
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅ (15,000+ tags)
Emotion Control	✅ (fine-grained inline control)
Languages	80+ (Tier 1: En, Zh, Jp)
Streaming	✅ (RTF 0.195, 100ms TTFA)
Model Size	~10 GB (BF16)
License	Fish Audio Research License

Links:

KittenTTS

KittenTTS

Description: KittenTTS is an open-source realistic text-to-speech model designed for lightweight deployment. It is a state-of-the-art TTS model under 25MB with just 15 million parameters, running without GPU on any device.

Release Date: February 24, 2026 (v0.8.1)

Feature	Value
Parameters	15M-80M
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	✅
Languages	English, Multiple
Streaming	✅
License	Apache-2.0

Links:

MOSS-TTS

MOSS-TTS

Description: MOSS-TTS is a production-grade Text-to-Speech foundation model developed by OpenMOSS Team and MOSI.AI. Features state-of-the-art evaluation performance on Seed-TTS-eval benchmark with zero-shot voice cloning.

Release Date: February 10, 2026

Feature	Value
Parameters	8B (Delay), 1.7B (Local)
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅ (Pinyin/Phoneme-level)
Emotion Control	✅
Languages	20 languages
Streaming	✅
Max Duration	1 hour
License	Apache-2.0

Links:

SoulX-Singer

SoulX-Singer

Description: SoulX-Singer is a high-fidelity, zero-shot singing voice synthesis model for generating realistic singing voices for unseen singers without fine-tuning.

Release Date: February 6, 2026

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅ (Singing)
ASR	❌
Pronunciation Control	✅ (MIDI/F0)
Emotion Control	✅
Languages	Mandarin, English, Cantonese
Streaming	✅
License	Apache-2.0

Links:

SoproTTS

SoproTTS

Description: SoproTTS is a lightweight English text-to-speech model with zero-shot voice cloning. It uses dilated convolutions (WaveNet-style) and lightweight cross-attention layers instead of the common Transformer architecture.

Release Date: February 4, 2026 (v1.5)

Feature	Value
Parameters	135M
Zero-shot Voice Cloning	✅ (3-12s)
ASR	❌
Pronunciation Control	❌
Emotion Control	✅ (style_strength)
Languages	English
Streaming	✅ (250ms TTFA)
RTF	0.05 (CPU M3)
Training Cost	~$100
License	Apache-2.0

Links:

NeuTTS

NeuTTS

Description: NeuTTS is a collection of open-source on-device TTS models with instant voice cloning. Built off LLM backbones with GGUF format quantizations for efficient on-device deployment.

Release Date: Early 2026

Feature	Value
Parameters	360M (Air), 120M (Nano)
Zero-shot Voice Cloning	✅ (3-second)
ASR	❌
Pronunciation Control	❌
Emotion Control	❌
Languages	English, Spanish, German, French
Streaming	✅
On-Device	✅ (GGUF quantizations)
License	Apache-2.0 (Air), NeuTTS Open License 1.0 (Nano)

Links:

Qwen3-TTS

Qwen3-TTS

Description: Qwen3-TTS is an open-source series of Text-to-Speech models developed by Alibaba Cloud. Supports stable, expressive, and streaming speech generation with free-form voice design.

Release Date: January 22, 2026

Feature	Value
Parameters	0.6B-1.7B
Zero-shot Voice Cloning	✅ (3-second)
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	10 (Chinese, English, Japanese, Korean, German, French, Russian, Portuguese, Spanish, Italian)
Streaming	✅ (97ms latency)
License	Apache-2.0

Links:

GLM-TTS

GLM-TTS

Description: High-quality TTS synthesis system based on LLMs from ZhipuAI, supporting zero-shot voice cloning with Multi-Reward Reinforcement Learning.

Release Date: December 11, 2025

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅ (3-10s)
ASR	❌
Pronunciation Control	✅ (Phoneme-level)
Emotion Control	✅ (RL-enhanced)
Languages	Chinese, English
Streaming	✅
License	Apache-2.0

Links:

VibeVoice-Realtime

VibeVoice-Realtime

Description: Real-time TTS model from Microsoft with streaming text input and ultra-low latency (~300ms).

Release Date: December 3, 2025

Feature	Value
Parameters	0.5B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	Multilingual
Streaming	✅ (300ms)
Max Duration	~10 minutes
License	MIT

Links:

Fun-CosyVoice 3.0

Fun-CosyVoice 3.0

Description: Advanced TTS system based on LLMs for zero-shot multilingual speech synthesis from FunAudioLLM.

Release Date: December 2025

Feature	Value
Parameters	0.5B
Zero-shot Voice Cloning	✅ (Multi-lingual/Cross-lingual)
ASR	❌
Pronunciation Control	✅ (Pinyin/CMU)
Emotion Control	✅
Languages	9 + 18+ Chinese dialects
Streaming	✅ (150ms)
License	Apache-2.0

Links:

MioTTS-2.6B

MioTTS-2.6B

Description: Lightweight, high-speed LLM-based TTS model for English and Japanese with minimal resource usage.

Release Date: 2026

Feature	Value
Parameters	2.6B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	❌
Languages	English, Japanese
Streaming	✅
RTF	0.135-0.145
License	LFM Open License

Links:

Supertonic 2

Supertonic 2

Description: Lightning-fast, on-device text-to-speech system designed for extreme performance with minimal computational overhead. Powered by ONNX Runtime, it runs entirely on-device—no cloud, no API calls, no privacy concerns. Outperforms ElevenLabs Flash v2.5 by up to 42× in speed benchmarks.

Release Date: 2026

Feature	Value
Parameters	66M
Zero-shot Voice Cloning	❌
ASR	❌
Pronunciation Control	❌
Emotion Control	❌
Languages	English, Korean, Spanish, Portuguese, French
Streaming	✅
RTF	0.001-0.015 (up to 167× realtime)
On-Device	✅ (ONNX Runtime)
License	OpenRAIL-M

Performance Comparison:

System	Speed (chars/sec)	RTF
Supertonic 2 (RTX 4090)	12,164	0.001
Supertonic 2 (M4 Pro CPU)	1,263	0.012
ElevenLabs Flash v2.5	287	0.5
Kokoro (Open-source)	117	1.3

Links:

KugelAudio

KugelAudio

Description: Open-source TTS for European languages with 7B parameters. Outperformed ElevenLabs in human preference testing.

Release Date: Early 2026

Feature	Value
Parameters	7B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅ (Speaking styles)
Languages	23 European languages
Streaming	✅
License	MIT

Links:

Kokoro-82M

Kokoro-82M

Description: Kokoro is an open-weight Text-to-Speech model with 82 million parameters. Despite its lightweight architecture, it delivers comparable quality to larger models while being significantly faster and more cost-efficient. With Apache-licensed weights, Kokoro can be deployed anywhere from production environments to personal projects.

Release Date: January 27, 2025 (v1.0)

Feature	Value
Parameters	82M
Architecture	StyleTTS 2, ISTFTNet
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅ (via misaki G2P)
Emotion Control	✅ (voice styles)
Languages	8 (54 voices)
Streaming	✅ (generator pattern)
Cost	<$0.06 per hour of audio
License	Apache-2.0

Links:

KokoClone

KokoClone

Description: KokoClone is a fast, real-time compatible multilingual voice cloning system built on top of Kokoro-ONNX. It enables users to type text in multiple languages, provide a short 3-10 second reference audio clip, and instantly generate speech in that same voice.

Release Date: 2025

Feature	Value
Parameters	82M (Base: Kokoro-ONNX)
Zero-shot Voice Cloning	✅ (3-10s reference)
ASR	❌
Pronunciation Control	❌
Emotion Control	✅
Languages	7 (En, Hi, Fr, Ja, Zh, It, Pt, Es)
Streaming	✅ (CPU real-time)
License	Apache-2.0

Links:

IndexTTS2

IndexTTS2

Description: AI-Enhanced Text-to-Speech System with Intelligent Optimization and self-learning capabilities.

Release Date: November 2025

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅ (5 emotions)
Languages	Chinese, English
Streaming	✅
Multi-speaker	✅ (1-4 speakers)
License	Apache-2.0

Links:

Maya1

Maya1

Description: State-of-the-art speech model for expressive voice generation with natural language voice control.

Release Date: November 2025

Feature	Value
Parameters	3B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅ (Tags)
Languages	English (Multi-accent)
Streaming	✅ (<100ms)
License	Apache-2.0

Links:

LFM2-Audio-1.5B

LFM2-Audio-1.5B

Description: Liquid AI's first end-to-end audio foundation model with low latency and real-time conversation.

Release Date: November 28, 2025

Feature	Value
Parameters	1.5B
Zero-shot Voice Cloning	✅
ASR	✅ (Integrated)
Pronunciation Control	N/A
Emotion Control	✅
Languages	English
Streaming	✅
License	LFM Open License

Links:

Step-Audio-EditX

Step-Audio-EditX

Description: 3B-parameter LLM-based RL audio model specialized in expressive and iterative audio editing.

Release Date: November 2025

Feature	Value
Parameters	3B (4B BF16)
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅ (Polyphone)
Emotion Control	✅ (14 emotions)
Languages	Mandarin, English, Sichuanese, Cantonese, Japanese, Korean
Streaming	✅
License	Apache-2.0

Links:

FireRedTTS2

FireRedTTS2

Description: Long-form streaming TTS system for multi-speaker dialogue generation with stable, natural speech.

Release Date: September 2025

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	EN, ZH, JP, KO, FR, DE, RU
Streaming	✅ (140ms)
Multi-speaker	✅ (4 speakers)
Max Duration	3 minutes
License	Apache-2.0

Links:

VoxCPM

VoxCPM

Description: Tokenizer-free TTS system for context-aware speech generation and true-to-life voice cloning.

Release Date: September 16, 2025

Feature	Value
Parameters	640M-800M
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	Chinese, English
Streaming	✅ (RTF 0.17)
License	Apache-2.0

Links:

LuxTTS

LuxTTS

Description: Lightweight ZipVoice-based TTS model for high quality voice cloning at speeds exceeding 150x realtime.

Release Date: 2025

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	❌
Emotion Control	❌
Languages	-
Streaming	✅
RTF	150x
VRAM	1GB
License	Apache-2.0

Links:

MegaTTS3

MegaTTS3

Description: Advanced zero-shot speech synthesis with Sparse Alignment Enhanced Latent Diffusion Transformer.

Release Date: March 22, 2025

Feature	Value
Parameters	0.45B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	Chinese, English
Streaming	✅
License	Apache-2.0

Links:

Spark-TTS

Spark-TTS

Description: Efficient LLM-Based TTS Model with Single-Stream Decoupled Speech Tokens, built on Qwen2.5.

Release Date: March 2025

Feature	Value
Parameters	0.5B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	Chinese, English
Streaming	✅
License	Apache-2.0

Links:

Fish Speech

Fish Speech

Description: State-of-the-art open source TTS and voice cloning model that generates natural, realistic, and emotionally rich speech.

Release Date: May 31, 2025 (v1.5.1)

Feature	Value
Parameters	4B (S1), 0.5B (S1-mini)
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	8 (EN, JP, KO, ZH, FR, DE, AR, ES)
Streaming	✅
RTF	~1:7
License	Apache-2.0

Links:

Step-Audio

Step-Audio

Description: Production-ready open-source framework for intelligent speech interaction with unified speech comprehension and generation.

Release Date: February 17, 2025

Feature	Value
Parameters	130B (Chat), 3B (TTS)
Zero-shot Voice Cloning	✅
ASR	✅
Pronunciation Control	✅
Emotion Control	✅
Languages	Chinese, English, Japanese
Streaming	✅
License	Apache-2.0

Links:

Audio Flamingo 3 (AF3) / Audio Flamingo Next

Audio Flamingo 3 (AF3) / Audio Flamingo Next

Description: NVIDIA ADLR's fully open-source Large Audio Language Model with state-of-the-art audio understanding. Audio Flamingo Next (AF-Next) is the latest generation featuring stronger general audio understanding, longer context support, and timestamp-grounded reasoning.

Release Date: July 2025 (AF3), 2026 (AF-Next)

Feature	Value
Parameters	7B
Zero-shot Voice Cloning	❌
ASR	✅
Pronunciation Control	N/A
Emotion Control	✅
Languages	Multi-lingual
Streaming	✅
Context	Up to 30 minutes
License	Apache-2.0

Key Innovation (AF-Next): Staged curriculum training with GRPO-based RL post-training. Three specialized checkpoints: Instruct, Think (reasoning), and Captioner. Temporal Audio Chain-of-Thought grounding intermediate reasoning to timestamps.

Links:

SoulX-Podcast

SoulX-Podcast

Description: SOTA Multi-Speaker TTS model for generating realistic long-form podcasts with dialectal diversity.

Release Date: 2025

Feature	Value
Parameters	-
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	Mandarin, English, Cantonese, Sichuanese, Henanese
Streaming	✅
Max Duration	90+ minutes
License	Apache-2.0

Links:

Chatterbox

Chatterbox

Description: Family of SOTA open-source TTS models by Resemble AI with zero-shot voice cloning and multilingual synthesis.

Release Date: June 13, 2025 (v0.1.2)

Feature	Value
Parameters	350M-500M
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅ (Tags)
Languages	23+
Streaming	✅
License	MIT

Links:

Orpheus-TTS

Orpheus-TTS

Description: SOTA open-source TTS built on Llama-3b backbone demonstrating emergent capabilities of LLMs for speech synthesis.

Release Date: April 2025

Feature	Value
Parameters	3B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	Multilingual
Streaming	✅ (200ms)
License	Apache-2.0

Links:

Dia

Dia

Description: 1.6B parameter TTS model by Nari Labs for generating ultra-realistic dialogue in one pass.

Release Date: June 27, 2024

Feature	Value
Parameters	1.6B
Zero-shot Voice Cloning	✅
ASR	❌
Pronunciation Control	✅
Emotion Control	✅
Languages	English
Streaming	✅
License	Apache-2.0

Links:

VieNeu-TTS

VieNeu-TTS

Description: Advanced on-device Vietnamese TTS model with instant voice cloning from 3-5 seconds of reference audio.

Release Date: 2025

Feature	Value
Parameters	0.3B-0.6B
Zero-shot Voice Cloning	✅ (3-5s)
ASR	❌
Pronunciation Control	✅
Emotion Control	❌
Languages	Vietnamese
Streaming	✅ (On-device)
License	Apache-2.0

Links:

MiMo-Audio

MiMo-Audio

Description: Audio Language Model by Xiaomi functioning as a Few-Shot Learner with SOTA audio understanding.

Release Date: 2025

Feature	Value
Parameters	7B
Zero-shot Voice Cloning	✅
ASR	✅
Pronunciation Control	N/A
Emotion Control	✅
Languages	Multi-lingual
Streaming	✅
License	Apache-2.0

Links:

Kimi-Audio

Kimi-Audio

Description: Open-source audio foundation model by Moonshot AI for audio understanding, generation, and conversation.

Release Date: 2024

Feature	Value
Parameters	7B
Zero-shot Voice Cloning	✅
ASR	✅
Pronunciation Control	N/A
Emotion Control	✅
Languages	Multi-lingual
Streaming	✅
License	MIT/Apache-2.0

Links:

ZipVoice

ZipVoice

Description: Fast and high-quality zero-shot TTS models based on flow matching.

Release Date: June 16, 2025

Feature	Value
Parameters	123M
Zero-shot Cloning	✅
Languages	Chinese, English
Dialogue	✅
License	Apache-2.0

Links:

---

Music Generation Models

Music Quick Comparison

Model	Music Gen	Languages	Streaming	License
ACE-Step 1.5	✅	50+	✅	MIT
LeVo 2	✅	Zh/En	❌	Apache-2.0
Foundation-1	✅ (Samples)	-	❌	Stability AI
Music Flamingo	❌	-	-	Apache-2.0
Magenta Realtime	✅	-	✅	Apache-2.0/CC-BY-4.0
Uni-MoE (Audio)	✅	-	✅	Apache-2.0

ACE-Step 1.5

ACE-Step 1.5

Description: The most powerful local music generation model outperforming most commercial alternatives. Supports Mac, AMD, Intel, and CUDA devices.

Release Date: February 20, 2026 (v0.1.2)

Feature	Value
Parameters	0.6B-4B (LM), DiT variants
Music Generation	✅
Lyrics Support	✅ (50+ languages)
Voice2BGM	✅
Reference Audio	✅
Track Separation	✅
Duration	10s - 10min
VRAM	<4GB
Platforms	CUDA, MPS, ROCm, XPU, CPU
License	MIT

Links:

LeVo 2

LeVo 2 (SongGeneration 2)

Description: Open-source foundation model for commercial-grade music generation by Tencent AI Lab. It outperforms open-source baselines and rivals commercial systems in Overall Quality, Melody, Arrangement, Sound Quality, and Structure.

Release Date: 2025

Feature	Value
Architecture	Hybrid LLM-Diffusion
Music Generation	✅
Lyrics Support	✅ (Chinese, English)
Multilingual	✅ (Zh, En)
Text/Audio Prompts	✅
VRAM	12GB-22GB
License	Apache-2.0

Links:

Foundation-1

Foundation-1

Description: Structured text-to-sample generation model for music production workflows. Generates tempo-synced, key-aware, bar-aware sample generation with support for instrument identity, timbre control, and FX processing.

Release Date: 2025

Feature	Value
Type	Text-to-Sample (Music)
Base Model	stabilityai/stable-audio-open-1.0
Instrument Control	✅
Timbre Descriptors	✅ (Warm, Bright, etc.)
FX Tags	✅ (Reverb, Delay, etc.)
Musical Notation	✅ (Chord, Melody, Arp)
VRAM	~8GB
License	Stability AI Community License

Links:

Music Flamingo

Music Flamingo

Description: Large audio-language model designed to advance music (including song) understanding. Achieves SOTA on 10+ music benchmarks.

Release Date: 2025

Feature	Value
Parameters	-
Music Understanding	✅
Music Generation	❌
Rich Captions	✅
Music QA	✅
Reasoning	✅ (Chain-of-thought)
Long-form	✅
License	Apache-2.0

Links:

Magenta Realtime

Magenta Realtime

Description: Open music generation model from Google DeepMind enabling continuous generation of musical audio steered by text prompts or audio examples.

Release Date: August 2025

Feature	Value
Parameters	-
Music Generation	✅ (Real-time)
Text-to-Music	✅
Audio-to-Music	✅
Reference Audio	✅
Continuous Generation	✅
Latency	Style prompt 2s+
Context	10 seconds
Training Data	~190k hours
License	Apache-2.0 (code), CC-BY-4.0 (model)

Links:

SoulX-Singer

SoulX-Singer

(Already listed in TTS - singing voice synthesis)

Feature	Value
Parameters	-
Singing Generation	✅
Zero-shot	✅
Melody Control	✅ (F0/MIDI)
Languages	Mandarin, English, Cantonese
License	Apache-2.0

Uni-MoE (Audio)

Uni-MoE (Audio)

Description: MoE-based omnimodal model with voice cloning, TTS, T2M (text-to-music), and V2M (video-to-music).

Release Date: October 16, 2025 (Uni-MoE-Audio)

Feature	Value
Parameters	-
Voice Cloning	✅
TTS	✅
Text-to-Music	✅
Video-to-Music	✅
Dynamic Routing	✅
License	Apache-2.0

Links:

---

Anything to Audio

Models that can generate audio from multiple input modalities (video, text, image, audio). These are unified frameworks for multimodal audio synthesis.

Anything to Audio Quick Comparison

Model	Text	Video	Image	Audio	License
Woosh	✅	✅	❌	✅	Apache-2.0
PrismAudio	❌	✅	❌	❌	Apache-2.0
ThinkSound	✅	✅	❌	✅	Apache-2.0
HunyuanVideo-Foley	✅	✅	❌	❌	Research Only
MMAudio	✅	✅	✅	❌	Apache-2.0
AudioX	✅	✅	✅	✅	Apache-2.0
Uni-MoE (Audio)	✅	✅	❌	✅	Apache-2.0

AudioX / Audio-Omni

AudioX / Audio-Omni

Description: Audio-Omni is the first end-to-end framework unifying understanding, generation, and editing across general sound, music, and speech domains. Presented at SIGGRAPH 2026. AudioX is a unified framework integrating text, video, image, and audio conditions.

Release Date: March 2025 (AudioX), 2026 (Audio-Omni)

Feature	Value
Parameters	-
Text-to-Audio	✅
Text-to-Music	✅
Text-to-Speech	✅
Video-to-Audio/Music	✅
Audio Editing	✅ (Add/Remove/Extract/Style)
Voice Conversion	✅
License	Apache-2.0 / CC-BY-NC-4.0

Key Innovation: First unified framework covering all three audio domains. Combines frozen multimodal LLM (Qwen2.5-Omni) with trainable Diffusion Transformer for high-fidelity synthesis. Any-to-any audio processing.

Links:

MMAudio

MMAudio

Description: Multimodal joint training framework for high-quality synchronized audio generation from video and/or text inputs. State-of-the-art open source model for generating sounds for videos, images, and text prompts.

Release Date: December 2024 (CVPR 2025)

Feature	Value
Parameters	-
Video-to-Audio	✅
Text-to-Audio	✅
Image-to-Audio	✅
Synchronized Audio	✅
Multimodal Joint Training	✅
License	Apache-2.0

Links:

HunyuanVideo-Foley

HunyuanVideo-Foley

Description: Tencent's end-to-end video sound effect generation model for professional-grade AI Foley sound generation. Analyzes footage and creates immersive audio that matches the visual content perfectly.

Release Date: 2025

Feature	Value
Parameters	-
Video-to-Audio (Foley)	✅
Text-to-Audio	✅
High-Quality Foley	✅
Context-Aware	✅
Output Quality	48 kHz
License	Research & Non-commercial only

Links:

ThinkSound

ThinkSound

Description: Unified Any2Audio generation framework with flow matching guided by Chain-of-Thought (CoT) reasoning. Supports generating or editing audio from video, text, audio, or their combinations. Accepted to NeurIPS 2025.

Release Date: 2025

Feature	Value
Parameters	-
Video-to-Audio	✅ (SOTA)
Text-to-Audio	✅
Audio-to-Audio	✅
Audio Editing	✅
CoT-Driven Reasoning	✅
Interactive Object-centric Editing	✅
License	Apache-2.0 (Research only)

Links:

Woosh

Woosh

Description: Sony AI's sound effect foundation model for text-to-audio and video-to-audio generation. Includes Woosh-AE (audio encoder/decoder), Woosh-Flow/DFlow (T2A), and Woosh-VFlow/DVFlow (V2A) with distilled fast inference variants.

Release Date: 2026

Feature	Value
Text-to-Audio	✅
Video-to-Audio	✅
Audio Encoding	✅
Fast Inference	✅ (Distilled models)
Architecture	Flow-based generative models
License	Apache-2.0

Key Innovation: Optimized for sound effects (not general audio) with both public and private model versions. Video-conditioned generation without requiring captions. Competitive with Stable Audio Open and TangoFlux.

Links:

PrismAudio

PrismAudio

Description: Video-to-Audio generation framework with Reinforcement Learning and specialized Chain-of-Thought (CoT) planning. Decomposes reasoning into four specialized modules (Semantic, Temporal, Aesthetic, Spatial CoT) for comprehensive video understanding. Built upon ThinkSound.

Release Date: 2025 (ICLR 2026)

Feature	Value
Parameters	518M
Video-to-Audio	✅
CoT Planning	✅ (4 modules)
Multi-Dimensional RL	✅
Fast-GRPO	✅ (Hybrid ODE-SDE)
Inference Time	0.63 seconds
License	Apache-2.0

Performance Benchmarks:

Metric	VGGSound	AudioCanvas
Semantic (CLAP)	0.47	0.52
Temporal (DeSync↓)	0.41	0.36
Aesthetic (MOS-Q)	4.21±0.35	4.12±0.28

Links:

Uni-MoE (Audio)

Uni-MoE (Audio)

Description: MoE-based omnimodal model with voice cloning, TTS, T2M (text-to-music), and V2M (video-to-music).

Release Date: October 16, 2025 (Uni-MoE-Audio)

Feature	Value
Parameters	-
Voice Cloning	✅
TTS	✅
Text-to-Music	✅
Video-to-Music	✅
Dynamic Routing	✅
License	Apache-2.0

Links:

---

Audio Restoration & Enhancement

Audio Restoration & Enhancement Quick Comparison

Model	Type	Bandwidth Extension	Inpainting	License
NVIDIA A2SB	Restoration	✅	✅	NVIDIA Non-Commercial
NovaSR	Enhancement	✅	❌	Apache-2.0
AudioSR	Enhancement	✅	❌	MIT

NVIDIA A2SB

NVIDIA A2SB (Audio-to-Audio Schrodinger Bridges)

Description: Diffusion-based audio restoration model tailored for high-resolution music at 44.1kHz. An end-to-end, vocoder-free, multi-task model capable of both bandwidth extension (predicting high-frequency components) and inpainting (re-generating missing segments). Can restore hour-long audio inputs without boundary artifacts.

Release Date: January 2025

Feature	Value
Architecture	End-to-end vocoder-free
Bandwidth Extension	✅
Audio Inpainting	✅
High-Resolution	✅ (44.1kHz)
Long Audio	✅ (hour-long)
Streaming	❌
License	NVIDIA OneWay NonCommercial License

Links:

NovaSR

NovaSR

Description: Lightning fast audio upsampler - 50kB model that upscales 16kHz audio to 48kHz at 3500x realtime.

Release Date: 2025

Feature	Value
Size	52kB
Speed	3600x realtime (A100)
Input	16kHz
Output	48kHz
VRAM	Minimal
License	Apache-2.0

Links:

AudioSR

AudioSR

Description: Audio super resolution model using latent diffusion to upscale low-quality audio to 48kHz.

Release Date: February 12, 2026 (v1.1.1)

Feature	Value
Input	8kHz-48kHz
Output	48kHz
VRAM	6GB min
Stereo	✅
Long Audio	✅
License	MIT

Links:

---

Speech Recognition (ASR)

ASR Quick Comparison

Model	Languages	Streaming	License
Cohere Transcribe	14	✅	Apache-2.0
VibeVoice-ASR	50+	✅	MIT
FunASR	50+	✅	MIT

Cohere Transcribe

Cohere Transcribe

Description: Open-source automatic speech recognition (ASR) model developed by Cohere. A 2 billion parameter dedicated audio-in, text-out model that ranks #1 on the English ASR leaderboard.

Release Date: March 2026

Feature	Value
Parameters	2B
Architecture	Conformer-based encoder-decoder
ASR	✅
Languages	14 (En, Fr, De, It, Es, Pt, Gr, Nl, Pl, Zh, Jp, Ko, Vi, Ar)
Streaming	✅
RTFx	Up to 3x faster than comparable models
License	Apache-2.0

Key Features:

• Long-form transcription with automatic chunking (>35 seconds)
• Optional punctuation control
• Batched inference support
• vLLM integration for production serving
• Apple Silicon support via mlx-audio
• WebGPU browser deployment via transformers.js

Links:

VibeVoice-ASR

VibeVoice-ASR

Description: Microsoft's unified speech-to-text model for 60-minute long-form audio processing with speaker diarization and timestamping.

Release Date: January 21, 2026

Feature	Value
Parameters	7B
ASR	✅
Languages	50+
Streaming	✅
License	MIT

Links:

FunASR

FunASR

Description: Fundamental end-to-end speech recognition toolkit with SOTA pretrained models.

Release Date: Ongoing (First: 2023)

Feature	Value
ASR	✅
VAD	✅
Punctuation	✅
Speaker Diarization	✅
Multi-talker ASR	✅
Timestamp	✅
Emotion Recognition	✅
Languages	50+
License	MIT/Model License

Links:

---

Additional Resources

ComfyUI Integrations

Leaderboard

Resource	Description	Link
Open ASR Leaderboard	Hugging Face leaderboard for comparing ASR model performance across languages and metrics.

---

Contributing

This list is continuously evolving. If you have any models to add or updates to suggest, please feel free to contribute!

---

Last Updated: March 2026