Stars
Sound event localization, detection, and tracking of multiple overlapping and moving sources in 2D spherical space using convolutional recurrent neural network
Code for voicing silent speech from EMG. Official repository for the papers "Digital Voicing of Silent Speech" at EMNLP 2020 and "An Improved Model for Voicing Silent Speech" at ACL 2021. Also incl…
Amazon Nova Act is an AWS service for building and deploying highly reliable AI agents that automate UI-based workflows at scale.
This is the official implementation of our multi-channel multi-speaker multi-spatial neural audio codec architecture.
A Python Room Spatial Impulse Response Ray-Tracing Toolkit
When given different views of an object as input, it can tell us if that specific object is present in a larger picture or not.
SonicSim: A customizable simulation platform for speech processing in moving sound source scenarios
Fully open reproduction of DeepSeek-R1
A framework for few-shot evaluation of language models.
One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks
The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.
AIR-Bench: Benchmarking Large Audio-Language Models via Generative Comprehension
Training and evaluation pipeline for MEG and EEG brain signal encoding and decoding using deep learning. Code for our paper "Decoding speech perception from non-invasive brain recordings" published…
Official implementation of NeurIPS 2024 paper "DiffusionPDE: Generative PDE-Solving Under Partial Observation"
Impulse Response measurement tool for MATLAB
This is the official implementation of our mesh-based neural network (MESH2IR) to generate acoustic impulse responses (IRs) for indoor 3D scenes represented using a mesh.
This is the official implementation of our neural-network-based fast diffuse room impulse response generator (FAST-RIR) for generating room impulse responses (RIRs) for a given acoustic environment.
This is the official implementation of reverberant speech to room impulse response estimator
Expressive Anechoic Recordings of Speech (EARS)
Official code for SEE-2-SOUND: Zero-Shot Spatial Environment-to-Spatial Sound
Official release of the Eyeful Tower dataset, a high-fidelity multi-view capture of 11 real-world scenes, from the paper “VR-NeRF High-Fidelity Virtualized Walkable Spaces” (Xu et al., SIGGRAPH Asi…
This is the official implementation of our end-to-end binaural audio rendering approach (Listen2Scene) for virtual reality (VR) and augmented reality (AR) applications.
A Differentiable Room Acoustics Simulator
PyTorch Implementation of FastDiff (IJCAI'22)
Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Real Acoustic Fields An Audio-Visual Room Acoustics Dataset and Benchmark
WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)
PyTorch code and models for V-JEPA self-supervised learning from video.