Thanks to visit codestin.com
Credit goes to github.com

Skip to content

fix the s3prl frontend gradient backprop bug, ensuring feature_grad_mult=1.0#5297

Merged
mergify[bot] merged 2 commits intoespnet:masterfrom
simpleoier:joint_train_fix
Jul 21, 2023
Merged

fix the s3prl frontend gradient backprop bug, ensuring feature_grad_mult=1.0#5297
mergify[bot] merged 2 commits intoespnet:masterfrom
simpleoier:joint_train_fix

Conversation

@simpleoier
Copy link
Collaborator

Fix the feature_grad_mult for s3prl frontend.

  • When feature_grad_mult is set to 0.0 by s3prl, the if clause will not be executed.
  • Sometime this if clause can still be executed when feature_grad_mult=0.0, due to float(0.0) precision issue.
    @Emrys365 @YoshikiMas

@mergify mergify bot added the ESPnet2 label Jul 12, 2023
@YoshikiMas
Copy link
Contributor

I can observe clear differences in backward_time and grad_norm in the 1-1989 batch, with this change. I will be able to check the 1st epoch performance within a few days.

@YoshikiMas
Copy link
Contributor

I get the accuracy of the first epoch that is comparable with that with the older version of S3PRL while causing artifacts in enhanced speech.

@simpleoier
Copy link
Collaborator Author

@YoshikiMas Thank you!

@sw005320 sw005320 added Bugfix SSL self-supervised learning labels Jul 20, 2023
@sw005320 sw005320 added this to the v.202307 milestone Jul 20, 2023
@sw005320
Copy link
Contributor

@YoshikiMas and @simpleoier, can I merge this PR?

@sw005320 sw005320 added the auto-merge Enable auto-merge label Jul 21, 2023
@codecov
Copy link

codecov bot commented Jul 21, 2023

Codecov Report

Merging #5297 (d55a1b8) into master (0971f17) will not change coverage.
The diff coverage is 0.00%.

@@           Coverage Diff           @@
##           master    #5297   +/-   ##
=======================================
  Coverage   76.10%   76.10%           
=======================================
  Files         658      658           
  Lines       59156    59156           
=======================================
  Hits        45022    45022           
  Misses      14134    14134           
Flag Coverage Δ
test_integration_espnet1 65.96% <ø> (ø)
test_integration_espnet2 47.51% <0.00%> (-0.01%) ⬇️
test_python 66.49% <0.00%> (ø)
test_utils 23.17% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
espnet2/asr/frontend/s3prl.py 77.27% <0.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@mergify mergify bot merged commit ff427c3 into espnet:master Jul 21, 2023
@simpleoier simpleoier deleted the joint_train_fix branch March 28, 2024 03:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge Enable auto-merge Bugfix ESPnet2 SSL self-supervised learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants