Thanks to visit codestin.com
Credit goes to github.com

Skip to content

re-fix norm compatibility in scale discriminator #5249

Merged
kan-bayashi merged 2 commits intoespnet:masterfrom
kan-bayashi:fix/fix-norm-issue-in-scale-discriminator
Jun 23, 2023
Merged

re-fix norm compatibility in scale discriminator #5249
kan-bayashi merged 2 commits intoespnet:masterfrom
kan-bayashi:fix/fix-norm-issue-in-scale-discriminator

Conversation

@kan-bayashi
Copy link
Member

@kan-bayashi kan-bayashi commented Jun 22, 2023 •
edited
Loading

What?

Why?

When using load_pretrained_model, the state_dict of destination is updated with that of source.

espnet/espnet2/torch_utils/load_pretrained_model.py

Line 114 in 161e4bb

dst_state.update(src_state)

Therefore, when there is key mismatch, the final state dict includes keys included in both destination and source dict.
As a result, the state_dict at this point includes weight from source dict and weight_g / weight_v from destination dict:

(Pdb) state_dict.keys()
dict_keys(['discriminator.msd.discriminators.0.layers.0.0.bias', 'discriminator.msd.discriminators.0.layers.0.0.weight_orig', 'discriminator.msd.discriminators.0.layers.0.0.weight_u', 'discriminator.msd.discriminators.0.layers.0.0.weight_v', 'discriminator.msd.discriminators.0.layers.1.0.bias', 'discriminator.msd.discriminators.0.layers.1.0.weight_orig', 'discriminator.msd.discriminators.0.layers.1.0.weight_u', 'discriminator.msd.discriminators.0.layers.1.0.weight_v', 'discriminator.msd.discriminators.0.layers.2.0.bias', 'discriminator.msd.discriminators.0.layers.2.0.weight_orig', 'discriminator.msd.discriminators.0.layers.2.0.weight_u', 'discriminator.msd.discriminators.0.layers.2.0.weight_v', 'discriminator.msd.discriminators.0.layers.3.0.bias', 'discriminator.msd.discriminators.0.layers.3.0.weight_orig', 'discriminator.msd.discriminators.0.layers.3.0.weight_u', 'discriminator.msd.discriminators.0.layers.3.0.weight_v', 'discriminator.msd.discriminators.0.layers.4.0.bias', 'discriminator.msd.discriminators.0.layers.4.0.weight_orig', 'discriminator.msd.discriminators.0.layers.4.0.weight_u', 'discriminator.msd.discriminators.0.layers.4.0.weight_v', 'discriminator.msd.discriminators.0.layers.5.0.bias', 'discriminator.msd.discriminators.0.layers.5.0.weight_orig', 'discriminator.msd.discriminators.0.layers.5.0.weight_u', 'discriminator.msd.discriminators.0.layers.5.0.weight_v', 'discriminator.msd.discriminators.0.layers.6.0.bias', 'discriminator.msd.discriminators.0.layers.6.0.weight_orig', 'discriminator.msd.discriminators.0.layers.6.0.weight_u', 'discriminator.msd.discriminators.0.layers.6.0.weight_v', 'discriminator.msd.discriminators.0.layers.7.bias', 'discriminator.msd.discriminators.0.layers.7.weight_orig', 'discriminator.msd.discriminators.0.layers.7.weight_u', 'discriminator.msd.discriminators.0.layers.7.weight_v', 'discriminator.msd.discriminators.0.layers.0.0.weight', 'discriminator.msd.discriminators.0.layers.1.0.weight', 'discriminator.msd.discriminators.0.layers.2.0.weight', 'discriminator.msd.discriminators.0.layers.3.0.weight', 'discriminator.msd.discriminators.0.layers.4.0.weight', 'discriminator.msd.discriminators.0.layers.5.0.weight', 'discriminator.msd.discriminators.0.layers.6.0.weight', 'discriminator.msd.discriminators.0.layers.7.weight'])

As a result, the loading is failed. This PR fixes this issue.

Related issues

@kan-bayashi kan-bayashi added this to the v.202307 milestone Jun 22, 2023
@kan-bayashi kan-bayashi force-pushed the fix/fix-norm-issue-in-scale-discriminator branch from 271aa23 to b685366 Compare June 22, 2023 14:42
@codecov
Copy link

codecov bot commented Jun 23, 2023 •
edited
Loading

Codecov Report

Merging #5249 (b685366) into master (b5a88e9) will increase coverage by 3.61%.
The diff coverage is 25.00%.

@@            Coverage Diff             @@
##           master    #5249      +/-   ##
==========================================
+ Coverage   71.37%   74.99%   +3.61%     
==========================================
  Files         655      655              
  Lines       58546    58552       +6     
==========================================
+ Hits        41789    43909    +2120     
+ Misses      16757    14643    -2114
Flag Coverage Δ
test_integration_espnet1 66.24% <ø> (+<0.01%) ⬆️
test_integration_espnet2 47.64% <25.00%> (?)
test_python 65.27% <0.00%> (-0.01%) ⬇️
test_utils 23.27% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
espnet2/gan_tts/hifigan/hifigan.py 82.42% <25.00%> (-0.84%) ⬇️

... and 62 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@kan-bayashi kan-bayashi merged commit e741743 into espnet:master Jun 23, 2023
BriansIDP pushed a commit to BriansIDP/espnet that referenced this pull request Jun 26, 2023
@kan-bayashi
re-fix norm compatibility in scale discriminator (espnet#5249)
46eafca
@Waggles77 Waggles77 mentioned this pull request Jun 13, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

Bugfix ESPnet2 README

Projects

None yet

Milestone

v.202307

Development

Successfully merging this pull request may close these issues.

1 participant

@kan-bayashi