DiffFx

A PyTorch-based library for differentiable audio effects processing.

Note: Several excellent libraries already exist, such as GRAFX, dasp-pytorch, NablAFx, and torchcomp. Some of my code is inspired by these libraries, and I'm grateful to their developers for implementing several fundamental processors. My core extension will be developing human-interpretable effect processors, where the parameters of each processor can be easily understood by humans.

Overview

DiffFx provides a collection of differentiable audio effects processors that can be integrated into neural network architectures. Most implementations follow methods from books including Audio Effects: Theory, Implementation and Application and DAFX - Digital Audio Effects (Second Edition)

Installation

pip install diffFx-pytorch

or

git clone https://github.com/ytsrt66589/diffFx-pytorch.git
cd diffFx-pytorch
pip install -e .

Quick Start

You can control each processor using either dsp_params or nn_params. dsp_params represents the exact DSP parameters used for each processor, while nn_params contains normalized parameters ranging from 0 to 1 that are internally mapped to the desired DSP parameters by each processor. dsp_params offers precise manual control over each processor, while nn_params provides learnable control for neural networks.

Using DSP Params

import torch
from diffFx_pytorch.processors.dynamics import Compressor

# Create a compressor
compressor = Compressor(sample_rate=44100)

# Process audio with direct DSP parameters
output = compressor(input_audio, dsp_params={
    'threshold_db': -20.0,
    'ratio': 4.0,
    'knee_db': 6.0,
    'attack_ms': 5.0,
    'release_ms': 50.0,
    'makeup_db': 0.0
})

Neural Network Integration

The library supports deep learning integration through normalized parameters:

import torch
import torch.nn as nn
from diffFx_pytorch.processors.dynamics import Compressor

# Create a neural network controller
class CompressorNet(nn.Module):
    def __init__(self, input_size, num_params):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(input_size, 32),
            nn.ReLU(),
            nn.Linear(32, num_params),
            nn.Sigmoid()  # Output in range [0,1]
        )
    
    def forward(self, x):
        return self.net(x)

# Initialize processor and network
comp = Compressor(sample_rate=44100)
num_params = comp.count_num_parameters()
controller = CompressorNet(input_size=16, num_params=num_params)

# Process audio with predicted parameters
features = torch.randn(batch_size, 16)
norm_params = controller(features)
output = comp(input_audio, nn_params=norm_params)

Examples

Understanding the sound characteristic of each processor

Check examples/processors/notebook to see how each processor affect sound.

Features

Implemented Effects 🎛️

• Utilities
  • [] Send
  • [] Mid/Side Processing
• Linear Gain
  • Gain: playground
• EQ
  • ToneStack: playground
  • Graphic Equalizer: playground
  • Parametric Equalizer: playground
• Dynamics
  • Compressor: playground
  • Multi-band Compressor: playground
  • Limiter: playground
  • Multi-band Limiter: playground
  • Expander: playground
  • [] Multi-band Expander
  • Noise Gate: playground
  • [] Multi-band Noise Gate
  • [] Deesser
• Delay
  • Basic Delay: playground
  • Feedback Basic Delay: playground
  • Slapback Delay: playground
  • Ping-pong Delay: playground
  • Multi-taps Delay: playground
• Spatial
  • Stereo Panning: playground
  • Stereo Widener: playground
  • Multi-band Stereo Widener: playground
  • Stereo Enhancer: playground
• Modulation
  • Chorus: playground
  • Multi-voice Chorus: playground
  • Stereo Chorus: playground
  • Flanger: playground
  • Feedback Flanger: playground
  • Stereo Flanger: playground
  • Phaser: playground
• Reverb
  • Noise Shape Reverb: playground
  • [] Feedback Delay Network (FDN)
• Distortion (Nonlinear)
  • TanH: playground
  • Hard/Soft/Double-Soft/Cubic/ArcTanh/Rectifier/Exponential Clipper: playground
  • Bit Crusher

To-do

• [] Delay-based effects should be implemented by surrogate delay for easy optimization

Contributing

Contributions are welcome! Please feel free to submit a Pull Request. Check the to-do list above for effects that haven't been implemented yet.

Citation

If you use diffFx-pytorch in your research, please cite:

@software{difffx_pytorch,
  title = {diffFx-pytorch: Differentiable Audio Effects Processing in PyTorch},
  author = {Yen-Tung Yeh},
  year = {2024},
  url = {https://github.com/ytsrt66589/difffx-pytorch}
}

License

This project is licensed under the Apache License - see the LICENSE file for details.