Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add integration tests for ESPnetEZ#5725

Merged
sw005320 merged 55 commits intoespnet:masterfrom
Masao-Someki:add_ez_tests
Nov 11, 2024
Merged

Add integration tests for ESPnetEZ#5725
sw005320 merged 55 commits intoespnet:masterfrom
Masao-Someki:add_ez_tests

Conversation

@Masao-Someki
Copy link
Contributor

@Masao-Someki Masao-Someki commented Mar 30, 2024

What?

Adds integration tests for ESPnet EZ

  • ASR1
    • basic
    • streaming
    • RNN-Transducer
    • ASR Transducer (standalone)
  • ASR2
  • TTS1
  • GAN-TTS1
  • ENH1
    • basic
    • data augment
    • with preprocess
    • dynamic mixing
  • ENH-TSE1
    • basic (No fine-tune)
    • unk_nspk
    • random enrollment
    • variable nspk random enrollment
  • ENH-ASR1
  • SSL1 (No fine-tune)
  • ST1
    • basic
    • streaming
  • SPK1
  • S2T1
  • S2ST1
  • LM1
  • SLU

Why?

To ensure that ESPnetEZ supports the same tasks as espnet2.

See also

#5584

- add VITS test
…ord argument 'text_name'`

- `text_name` has default value defined in `CommonPreprocessor_multi` class, so we can simply delete this line.
- If we have multiple speakers with enh_s2t task, we have `text_spk{n}` as input. These should be defined in the configuration as `text_name`, which cause the above error.
- When config for fine-tune does not contain `preprocessor_conf`, it was replaced with empty dict.

- After this fix, only when fine-tune config contains `preprocessor_conf` the values will be replaced with the new values.
- No finetune test for hubert, since there is no inference class found.
@mergify mergify bot added ESPnet2 CI Travis, Circle CI, etc labels Mar 30, 2024
@codecov
Copy link

codecov bot commented Mar 30, 2024

Codecov Report

Attention: Patch coverage is 10.71429% with 25 lines in your changes missing coverage. Please review.

Project coverage is 35.62%. Comparing base (3858d84) to head (b5cb974).
Report is 366 commits behind head on master.

Files Patch % Lines
espnetez/dataset.py 9.09% 10 Missing ⚠️
espnetez/trainer.py 9.09% 10 Missing ⚠️
espnet2/tasks/st.py 0.00% 3 Missing ⚠️
espnetez/config.py 33.33% 2 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3858d84) and HEAD (b5cb974). Click for more details.

HEAD has 25 uploads less than BASE
Flag BASE (3858d84) HEAD (b5cb974)
test_python_espnetez 8 1
test_utils 9 0
test_integration_espnetez 8 0
test_python_espnet2 1 0
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #5725       +/-   ##
===========================================
- Coverage   71.58%   35.62%   -35.96%     
===========================================
  Files         760      758        -2     
  Lines       69841    69825       -16     
===========================================
- Hits        49996    24878    -25118     
- Misses      19845    44947    +25102     
Flag Coverage Δ
test_integration_espnet1 62.92% <ø> (ø)
test_integration_espnetez ?
test_python_espnet1 18.20% <0.00%> (-0.01%) ⬇️
test_python_espnet2 ?
test_python_espnetez 13.81% <10.71%> (-0.14%) ⬇️
test_utils ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Masao-Someki
Copy link
Contributor Author

Hi, when I try to run the integration test for the gan-tts task, I encounter the following error. I'm using the egs2/mini_an4/tts/conf/train_vits_debug.yaml to test the training.
Does anyone know how to avoid the following error?

RuntimeError: Trying to backward through the graph a second time (or directly access saved tensors after they have already been freed).
Saved intermediate values of the graph are freed when you call .backward() or autograd.grad().
Specify retain_graph=True if you need to backward through the graph a second time or if you need to access saved tensors after calling backward.

@sw005320
Copy link
Contributor

sw005320 commented Apr 1, 2024

@kan-bayashi, can you help the issue?

@Masao-Someki Masao-Someki changed the title [WIP] Add integration tests for ESPnetEZ Add integration tests for ESPnetEZ Nov 9, 2024
@Masao-Someki Masao-Someki marked this pull request as ready for review November 9, 2024 04:38
Masao-Someki and others added 18 commits November 8, 2024 23:39
- Changed to the original value
- The CI issue comes from the config for new transducer task.
- I have changed the config name to support configuration test for new transducer test.
- Config test does not have test for GAN-TTS, and difficult to support. To avoid error in CI, the config was moved to child directory.
- `preprosessor` is removed from config, as `enh_asr1` recipe does not have this argument.
- ENH_ASR task does not have configuration for `preprocessor`, but has `use_preprocessor`.
- We need to define `text_name` separately for this task, whereas other tasks defines this argument in `preprocessor_conf.text_name`
@sw005320
Copy link
Contributor

Thanks!
This looks very good to me!

@sw005320 sw005320 merged commit ba092ad into espnet:master Nov 11, 2024
@Masao-Someki Masao-Someki deleted the add_ez_tests branch November 12, 2024 15:59
Shikhar-S pushed a commit to Shikhar-S/espnet that referenced this pull request Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants