Codestin Search App

Jungjee · 2023-05-25T01:26:49Z

This pr includes the data preparation part for ESPnet-spk project.
It cannot be used at current status. Following PR will be dealing with the training part.

Includes VoxCeleb download script (crawl from YouTube)

data preparation part done.

for more information, see https://pre-commit.ci

Emrys365

I think you need to include other files/directories in the TEMPLATE/sv1 folder, just like the other tasks. For example:

conf/
local/path.sh
README.md
db.sh -> ../../TEMPLATE/asr1/db.sh
sv.sh
pyscripts -> ../../TEMPLATE/asr1/pyscripts
setup.sh
steps -> ../../TEMPLATE/asr1/steps
utils -> ../../TEMPLATE/asr1/utils

egs2/voxceleb/sv1/sv.sh should be a symbolic link pointing to egs2/TEMPLATE/sv1/sv.sh.

egs2/voxceleb/sv1/db.sh

egs2/voxceleb/sv1/__init__.py

egs2/voxceleb/sv1/run.sh

Jungjee · 2023-05-25T02:56:13Z

Thanks @Emrys365 for your review :)

egs2/TEMPLATE/sv1/db.sh

sw005320 · 2023-05-31T13:05:52Z

Can you fix the CI error?
https://github.com/espnet/espnet/actions/runs/5075591243/jobs/9128549995?pr=5184#step:7:281

codecov · 2023-05-31T19:36:41Z

Codecov Report

Merging #5184 (2d89067) into master (e6a628e) will increase coverage by 10.18%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           master    #5184       +/-   ##
===========================================
+ Coverage   64.35%   74.54%   +10.18%     
===========================================
  Files         640      640               
  Lines       57057    57267      +210     
===========================================
+ Hits        36719    42688     +5969     
+ Misses      20338    14579     -5759

Flag	Coverage Δ
test_integration_espnet1	`66.28% <ø> (?)`
test_integration_espnet2	`47.58% <ø> (?)`
test_python	`65.28% <ø> (+0.92%)`	⬆️
test_utils	`23.28% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

see 199 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

sw005320 · 2023-05-31T19:41:38Z

This is good enough as a first commit.
I'll merge it after the CI is finished.

sw005320 · 2023-06-01T12:00:47Z

Can you fix https://github.com/espnet/espnet/actions/runs/5136094955/jobs/9242613162?pr=5184#step:12:104181 by following the instruction?

sw005320 · 2023-06-01T16:01:42Z

@Jungjee, sorry, I tried to fix the issues, but it turns out that I need to change egs2/accented_french_openslr57/asr1/cmd.sh, egs2/voxceleb/sv1/cmd.sh, (and maybe egs2/TEMPLATE/sv1/cmd.sh) from symlink to the normal file.
Can you do that for me?

Basically, cmd.sh and slurm.conf etc. could be used for the user by directly editing them.
So, it would be better to use the actual file instead of the symlink.

Jungjee · 2023-06-01T18:49:22Z

@sw005320 , sure, but the error you ask me to fix before is
Error: egs2/accented_french_openslr57/asr1/cmd.sh and egs2/TEMPLATE/sv1/cmd.sh differ: To solve: for f in egs2/*/*/cmd.sh; do cp egs2/TEMPLATE/asr1/cmd.sh ${f}; done

which means to changes local cmd.sh with the symlink of TEMPLATE one.

So my understanding is to undo this and make CI error come again.

Pls let me know if I'm thinking something wrong.

Meanwhile, I cannot pull from Jungjee/master somehow, trying to solve this issue.

make cmd.sh into real files

sw005320 · 2023-06-01T18:56:40Z

which means to changes local cmd.sh with the symlink of TEMPLATE one.

First, egs2/TEMPLATE/asr1/cmd.sh is an actual file.
Not a symlink.
Then, do cp egs2/TEMPLATE/asr1/cmd.sh ${f} means that it just copies an actual file.
So, cmd.sh under the recipe directory should be an actual file.

Jungjee · 2023-06-01T19:01:06Z

Sure, I think it's okay now? after commit 46c17d2

sw005320 · 2023-06-01T19:02:53Z

Thanks, it should be fine, and let's ask the CI for the rest.

initial upload

sw005320 · 2023-06-02T11:30:10Z

Another issue
https://github.com/espnet/espnet/actions/runs/5150271646/jobs/9274249165?pr=5184#step:12:103790
I recommend you run the integration test locally by yourself to fix this in advance.

Speaker

add conf to template

add local

egs2/TEMPLATE/sv1/setup.sh

sw005320 · 2023-06-02T23:35:59Z

Can you remove some files under TEMPLATE/sv1/local?
We should not put recipe-related files under the template.
Please check https://github.com/espnet/espnet/tree/master/egs2/TEMPLATE/asr1/local

sw005320 · 2023-06-02T23:37:39Z

I just did it by myself.

Jee-weon Jung and others added 3 commits May 22, 2023 15:32

initial upload

d86576c

adapt to kaldi-style formatting

ddcbba2

Merge pull request #1 from Jungjee/speaker

1df57c6

data preparation part done.

mergify bot added the ESPnet2 label May 25, 2023

pre-commit-ci bot and others added 2 commits May 25, 2023 01:27

[pre-commit.ci] auto fixes from pre-commit.com hooks

ecd8fd4

for more information, see https://pre-commit.ci

Merge branch 'master' into master

da74017

Emrys365 reviewed May 25, 2023

View reviewed changes

ops: move to TEMPLATE

c7c59ac

Emrys365 reviewed May 25, 2023

View reviewed changes

egs2/voxceleb/sv1/db.sh Outdated Show resolved Hide resolved

egs2/voxceleb/sv1/__init__.py Outdated Show resolved Hide resolved

egs2/voxceleb/sv1/run.sh Outdated Show resolved Hide resolved

ftshijt added SID Speaker identification/embedding New Features labels May 25, 2023

reflect @Emrys365 comment

0190be2

ftshijt reviewed May 25, 2023

View reviewed changes

egs2/TEMPLATE/sv1/db.sh Show resolved Hide resolved

sw005320 added this to the v.202307 milestone May 31, 2023

Jee-weon Jung and others added 2 commits May 31, 2023 09:54

fix typo

f02e685

rm uncomplete file

fa74fa5

sw005320 added the auto-merge Enable auto-merge label May 31, 2023

fix accented_french_openslr57/asr1/cmd.sh rel lnik

58132d1

make cmd.sh into real files

46c17d2

Jungjee closed this Jun 1, 2023

Jungjee force-pushed the master branch from 15dd392 to 53ccddd Compare June 1, 2023 18:50

Merge pull request #7 from Jungjee/speaker

636755f

make cmd.sh into real files

Jungjee reopened this Jun 1, 2023

Merge branch 'espnet:master' into master

3d5205d

Jungjee and others added 2 commits June 1, 2023 19:38

initial upload

8ae838a

Merge pull request #8 from Jungjee/speaker

50edd44

initial upload

Jungjee and others added 7 commits June 2, 2023 15:33

add confs

6baed99

add necessary folders for ci

f83d811

Merge pull request #9 from Jungjee/speaker

96d6304

Speaker

add conf to template

6a3d910

Merge pull request #10 from Jungjee/speaker

cabeb6c

add conf to template

add local

59d2f80

Merge pull request #11 from Jungjee/speaker

83fd14f

add local

Emrys365 reviewed Jun 2, 2023

View reviewed changes

egs2/TEMPLATE/sv1/setup.sh Outdated Show resolved Hide resolved

egs2/TEMPLATE/sv1/setup.sh Outdated Show resolved Hide resolved

sw005320 added 4 commits June 2, 2023 19:36

Delete crawl_voxcelebs.py

4be7aab

Delete crawl_voxcelebs_mp.py

5bb6f5f

Delete data.sh

51aca22

Delete data_prep.py

2d89067

mergify bot merged commit d9ae3f7 into espnet:master Jun 3, 2023

Conversation

Jungjee commented May 25, 2023

Uh oh!

Emrys365 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jungjee commented May 25, 2023

Uh oh!

Uh oh!

sw005320 commented May 31, 2023

Uh oh!

codecov bot commented May 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sw005320 commented May 31, 2023

Uh oh!

sw005320 commented Jun 1, 2023

Uh oh!

sw005320 commented Jun 1, 2023

Uh oh!

Jungjee commented Jun 1, 2023

Uh oh!

sw005320 commented Jun 1, 2023

Uh oh!

Jungjee commented Jun 1, 2023

Uh oh!

sw005320 commented Jun 1, 2023

Uh oh!

sw005320 commented Jun 2, 2023

Uh oh!

Uh oh!

Uh oh!

sw005320 commented Jun 2, 2023

Uh oh!

sw005320 commented Jun 2, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

codecov bot commented May 31, 2023 •

edited

Loading