Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Update the chunk iterator for the TSE task#4929

Merged
mergify[bot] merged 9 commits intoespnet:masterfrom
Emrys365:tse
Mar 3, 2023
Merged

Update the chunk iterator for the TSE task#4929
mergify[bot] merged 9 commits intoespnet:masterfrom
Emrys365:tse

Conversation

@Emrys365
Copy link
Collaborator

@Emrys365 Emrys365 commented Feb 9, 2023

This PR updates the chunk iterator to make it more general for different tasks.

I added a new argument excluded_key_prefixes for ChunkIterFactory to allow certain keys to be ignored when checking the length consistency for each sample:

            for key in sequence_keys:
-               if len(batch[key]) != len(
-                   batch[sequence_keys[0]]
-               ) and not key.startswith("enroll_ref"):
+               if key.startswith(self.excluded_key_prefixes):
+                   # ignore length inconsistency for `excluded_key_prefixes`
+                   continue
+               if len(batch[key]) != len(batch[sequence_keys[0]]):
                    raise RuntimeError(
                        f"All sequences must has same length: "
                        f"{len(batch[key])} != {len(batch[sequence_keys[0]])}"

This can be useful for audio-to-audio tasks like target speaker extraction (TSE), where additional features are required as input, which do not necessarily have the same length as the input/target signal.

In our previous discussion in the SE meeting, we decided to make a new chunk iterator for such purpose. But after some coding, I find the current minimal changes in espnet2/iterators/chunk_iter_factory.py are not so bad.

If you still feel it necessary to make a new script, I will do it.

@Emrys365 Emrys365 added ESPnet2 SE Speech enhancement labels Feb 9, 2023
@codecov
Copy link

codecov bot commented Feb 9, 2023

Codecov Report

Merging #4929 (533b837) into master (1bed2f9) will increase coverage by 2.20%.
The diff coverage is 54.54%.

@@            Coverage Diff             @@
##           master    #4929      +/-   ##
==========================================
+ Coverage   74.79%   77.00%   +2.20%     
==========================================
  Files         606      606              
  Lines       53721    53761      +40     
==========================================
+ Hits        40183    41396    +1213     
+ Misses      13538    12365    -1173     
Flag Coverage Δ
test_integration_espnet1 66.33% <ø> (ø)
test_integration_espnet2 47.59% <54.54%> (+0.09%) ⬆️
test_python 66.84% <54.54%> (+2.46%) ⬆️
test_utils 23.35% <ø> (+0.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
espnet2/train/preprocessor.py 29.19% <0.00%> (-0.39%) ⬇️
espnet2/iterators/chunk_iter_factory.py 82.17% <100.00%> (+12.17%) ⬆️
espnet2/tasks/abs_task.py 78.25% <100.00%> (+2.57%) ⬆️
espnet2/train/trainer.py 76.86% <0.00%> (-0.47%) ⬇️
espnet2/uasr/espnet_model.py 0.00% <0.00%> (ø)
espnet/nets/pytorch_backend/e2e_vc_transformer.py 86.72% <0.00%> (+0.11%) ⬆️
espnet/nets/pytorch_backend/e2e_vc_tacotron2.py 80.48% <0.00%> (+0.15%) ⬆️
espnet/nets/chainer_backend/e2e_asr_transformer.py 69.59% <0.00%> (+0.20%) ⬆️
espnet/nets/pytorch_backend/lm/seq_rnn.py 86.88% <0.00%> (+0.21%) ⬆️
espnet2/bin/asr_transducer_inference.py 94.04% <0.00%> (+0.39%) ⬆️
... and 34 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

self.seed = seed
self.shuffle = shuffle
self.excluded_key_pattern = (
"(" + "[0-9]*)|(".join(excluded_key_prefixes) + "[0-9]*)"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you only assuming the cases of numbers are appended?
If so, it should be documented in add_argument or other places.
Adding a comment to the config is also informative.

Copy link
Collaborator Author

@Emrys365 Emrys365 Feb 18, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, only an exact match and those matched with trailing numbers are considered here. I will update the information in the argument definition and configs.

@sw005320 sw005320 added this to the v.202303 milestone Mar 3, 2023
@sw005320 sw005320 added the Refactoring Refactoring label Mar 3, 2023
@sw005320 sw005320 added the auto-merge Enable auto-merge label Mar 3, 2023
@mergify mergify bot merged commit a4aeeb2 into espnet:master Mar 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge Enable auto-merge ESPnet2 Refactoring Refactoring SE Speech enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants