Welcome to the official release of pretrained models and scripts for Nes2Net, a lightweight nested architecture designed for foundation model-driven speech anti-spoofing, as described in our paper

Accpeted to: IEEE Transactions on Information Forensics and Security (T-IFS), IEEE Link: https://ieeexplore.ieee.org/document/11222612

This repository supports the following datasets:

• ASVspoof 2019 / 2021
• ASVspoof 5
• In-the-Wild
• Controlled Singing Voice Deepfake Detection (CtrSVDD): View here

• Main Branch (Current Branch): For ASVspoof 2019/2021 and In-the-Wild datasets

• Asvspoof5 Branch: For the ASVspoof 5 dataset 👉 asvspoof5 branch

🔥[April 20, 2025] ASVspoof 5 pretrained model and scripts are now avaiable!

※: with extra data augmentation.

‡: ensemble of multiple models

※: with extra data augmentation.

• Only best model checkpoints are provided.

1. Git clone this repo.

2. Build environment:

   conda env create -f environment.yml
   

   or

   pip install -r requirements.txt
   

   • You may need to adjust the library versions according to your CUDA version and GPU spec.

3. Setup fairseq:

   cd fairseq-a54021305d6b3c4c5959ac9395135f63202db8f1
   (This fairseq folder can also be downloaded from https://github.com/pytorch/fairseq/tree/a54021305d6b3c4c5959ac9395135f63202db8f1)
   pip install --editable ./
   pip install -r requirements.txt
   

4. Pre-trained wav2vec 2.0 XLSR (300M): Download the XLSR models from here

If you want to train the model: The ASVspoof 2019 dataset can be downloaded from here.

If you want to test on the ASVspoof 2021 database, it is released on the zenodo site.

-- LA here

-- DF here

-- keys (labels) and metadata here

If you want to test on the In-the-Wild dataset, it can be downloaded from here

1. Download the pretrained checkpoints from above table Google Drive links. For example, WavLM_Nes2Net_X.
2. Run

   CUDA_VISIBLE_DEVICES=0 python easy_inference_demo.py \
   --model_path [pretrained_model_path] \
   --file_to_test [the file to test] \
   --model_name xxxx
   

   Following is an example:

   CUDA_VISIBLE_DEVICES=0 python easy_inference_demo.py \
   --model_path "/data/tianchi/Nes2Net_SVDD_ckpts/WavLM_Nes2Net_X_SeLU_e74_seed420_valid0.04245662278274772.pt" \
   --file_to_test "/home/tianchi/Nes2Net_ASVspoof_ITW/database/LA/ASVspoof2019_LA_dev/flac/LA_D_1000265.flac" \
   --model_name WavLM_Nes2Net_X
   

check the command template:

CUDA_VISIBLE_DEVICES=0 python main.py --track=LA --lr=0.00000025 --batch_size=12 \
--algo 4 --date 0520 --seed 12345 --loss=WCE --model_name wav2vec2_Nes2Net_X \
--num_epochs 100 --pool_func 'mean' --SE_ratio 1 --Nes_ratio 8 8 \
--database_path /home/tianchi/SSL_Anti-spoofing/database/LA/

• Change the --database_path to your ASVspoof dataset path.

check the command template in:

test.sh

1. For single model or averaged pretrained model: For the ASVspoof 21 LA, an example:

   python main.py --track=LA --batch_size=2 --is_eval --eval --model_name wav2vec2_Nes2Net_X --pool_func 'mean' --SE_ratio 1 --Nes_ratio 8 8 --test_protocol '4sec' \
   --database_path '/home/tianchi/database/LA/' \
   --model_path="./wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_ep95.pth" \
   --eval_output='score_LA_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_ep95.txt'
   

   For the ASVspoof 21 DF, change the above command with:

   --track=DF
   

   For already averaged model, simply change the model path:

   --model_path="./wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345_avg_ckpt_ep59_61_69.pth" \
   

   • Change the --database_path to your ASVspoof dataset path.
   • Change the --model_path to your path of the checkpoint to test. You may use the checkpoint with the smallest validation EER for testing.
   • if you wish to average checkpoints and then test, pls refer to point 3 below

   And then, to get the final result of EER and minDCF:

   txtpath=score_LA_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_ep95.txt
   python evaluate_2021_LA.py $txtpath /home/tianchi/database/keys eval
   

   and

   txtpath=score_DF_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_ep95.txt
   python evaluate_2021_DF.py $txtpath /home/tianchi/database/keys eval
   

2. For averaging checkpoints and then test, an example:

   python main.py --track=DF --batch_size=2 --is_eval --eval --model_name wav2vec2_Nes2Net_X --pool_func 'mean' --SE_ratio 1 --Nes_ratio 8 8 --test_protocol '4sec' \
   --num_average_model 5 --model_ID_to_average 56 60 62 76 95 \
   --database_path '/home/tianchi/database/LA/' \
   --model_folder_path="/home/tianchi/SSL_Anti-spoofing/models/wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4" \
   --eval_output='score_DF_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_avg_ckpt_ep56_60_62_76_95.txt'
   

   • If you use checkpoints average function, choose the serveral epochs with smallest validation EERs for testing according to validation EER, and set as --model_ID_to_average.
   • change the --model_folder_path to the path of the folder that saving all checkpoints.
   • Change the --pool_func --Nes_ratio --SE_ratio --model_name to match your training setting.
   • If you are using the pretrained model, these configs are the same as default.

   And then, similar as that of point 2 above, following the examples to get the EER and minDCF from the score text file.

1. For single model or averaged pretrained model: The same as the command above for ASVspoof 21LA and DF, just change the --track=in_the_wild and --in_the_wild_path, following is anexnample:

   python main.py --track=in_the_wild --batch_size=2 --is_eval --eval --model_name wav2vec2_Nes2Net_X --pool_func 'mean' --SE_ratio 1 --test_protocol 'full' \
   --in_the_wild_path='/home/tianchi/SSL_Anti-spoofing/database/release_in_the_wild' \
   --model_path="/home/tianchi/SSL_Anti-spoofing/models/wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345/wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345_avg_ckpt_ep59_61_69.pth" \
   --eval_output='score_ITW_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345_avg_ckpt_ep59_61_69.txt' \
   

2. Similarly, you can also average checkpoints and then test, an example:

   python main.py --track=in_the_wild --batch_size=2 --is_eval --eval --model_name wav2vec2_Nes2Net_X --pool_func 'mean' --SE_ratio 1 --test_protocol 'full' \
   --num_average_model 3 --model_ID_to_average 59 61 69 \
   --in_the_wild_path='/home/tianchi/SSL_Anti-spoofing/database/release_in_the_wild' \
   --model_folder_path="/home/tianchi/SSL_Anti-spoofing/models/wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345" \
   --eval_output='score_ITW_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345_avg_ckpt_ep59_61_69.txt' \
   

3. scoring:

   python Cal_EER_in_the_wild.py --path [path to score txt] 
   

   For example:

   python Cal_EER_in_the_wild.py --path score_ITW_wav2vec2_Nes2Net_X_e100_bz12_lr2.5e_07_algo4_seed12345_avg_ckpt_ep59_61_69.txt
   

Thanks for following open-source projects:

1. wav2vec2 + AASIST & Rawboost: https://github.com/TakHemlata/SSL_Anti-spoofing Paper: [model], [Rawboost]
2. SEA aggregation: https://github.com/Anmol2059/SVDD2024 Paper: [SEA]
3. AttM aggregation: https://github.com/pandarialTJU/AttM_INTERSPEECH24 Paper: [AttM]

@ARTICLE{Nes2Net,
  author={Liu, Tianchi and Truong, Duc-Tuan and Kumar Das, Rohan and Aik Lee, Kong and Li, Haizhou},
  journal={IEEE Transactions on Information Forensics and Security}, 
  title={Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-Spoofing}, 
  year={2025},
  volume={20},
  number={},
  pages={12005-12018},
  keywords={Foundation models;Feature extraction;Computational modeling;Computer architecture;Computational efficiency;Dimensionality reduction;Acoustics;Kernel;Robustness;Deepfakes;Deepfake detection;speech anti-spoofing;Res2Net;Nes2Net;SSL;speech foundation model},
  doi={10.1109/TIFS.2025.3626963}}