Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ESC-50 classification with BEATs#5977

Merged
sw005320 merged 33 commits intoespnet:masterfrom
Shikhar-S:esc
Jan 5, 2025
Merged

ESC-50 classification with BEATs#5977
sw005320 merged 33 commits intoespnet:masterfrom
Shikhar-S:esc

Conversation

@Shikhar-S
Copy link
Contributor

@Shikhar-S Shikhar-S commented Dec 10, 2024 •
edited
Loading

What?

  1. Adds a linear layer decoder for classification tasks.
  2. Adds classification recipe with ESC-50 dataset and BEATs features.
  3. Adds Roll Augmentation to ESPnet.

Why?

  1. Sets up the framework for classification tasks in ESPnet using asr1 recipe.
  2. This is an effort to reproduce results from the BEATs paper on classification datasets.

Status

Current results
Ours: 93.8% 94.8%
BEATs: 95.6 (Table 1 from this paper)
Edit: This difference means 3.2 additional examples are wrong (test set is of size 400). Not sure if this is significant.

Current setup is missing:

  1. Roll augmentation Added
  2. SpecAugment on frequency bands Tuned

See also

ESC-50 data prep code is adapted from #5410
Uses BEATs implementation from #5915

@mergify mergify bot added the ESPnet2 label Dec 10, 2024
@Shikhar-S Shikhar-S marked this pull request as ready for review December 10, 2024 16:43
@mergify mergify bot added the README label Dec 10, 2024
@codecov
Copy link

codecov bot commented Dec 10, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 28.43137% with 73 lines in your changes missing coverage. Please review.

Project coverage is 47.44%. Comparing base (ef6740c) to head (e4d2822).

Files with missing lines Patch % Lines
espnet2/asr/decoder/linear_decoder.py 26.66% 33 Missing ⚠️
espnet/nets/pytorch_backend/nets_utils.py 12.50% 14 Missing ⚠️
espnet2/asr/espnet_model.py 30.00% 14 Missing ⚠️
espnet2/asr/encoder/beats_encoder.py 11.11% 8 Missing ⚠️
espnet2/bin/asr_inference.py 63.63% 4 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #5977      +/-   ##
==========================================
- Coverage   47.48%   47.44%   -0.05%     
==========================================
  Files         529      530       +1     
  Lines       47850    47936      +86     
==========================================
+ Hits        22721    22742      +21     
- Misses      25129    25194      +65
Flag Coverage Δ
test_integration_espnet2 47.44% <28.43%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mergify mergify bot added the ESPnet1 label Dec 15, 2024
@Shikhar-S
Copy link
Contributor Author

Shikhar-S commented Dec 15, 2024

This is ready for review.

@sw005320 sw005320 added the SED Sound Event Detection label Dec 15, 2024
@sw005320 sw005320 added this to the v.202503 milestone Dec 15, 2024
@sw005320 sw005320 requested a review from ftshijt December 15, 2024 12:28
ftshijt
ftshijt reviewed Dec 27, 2024
View reviewed changes
Copy link
Collaborator

@ftshijt ftshijt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the update! I left some minor comments

@Shikhar-S
Copy link
Contributor Author

Shikhar-S commented Dec 29, 2024

Hi, should I be doing anything else on this PR? (The failing test seems to be due to time out, iiuc)

sw005320
sw005320 reviewed Jan 1, 2025
View reviewed changes
@Shikhar-S
Copy link
Contributor Author

Shikhar-S commented Jan 1, 2025

Thanks for reviewing this PR.

@sw005320
Copy link
Contributor

sw005320 commented Jan 1, 2025

Please fix this issue
https://github.com/espnet/espnet/actions/runs/12572260233/job/35044059820?pr=5977

sw005320
sw005320 reviewed Jan 2, 2025
View reviewed changes
@Shikhar-S Shikhar-S mentioned this pull request Jan 2, 2025
Merged
@Shikhar-S Shikhar-S requested a review from sw005320 January 5, 2025 16:52
@sw005320 sw005320 merged commit b70dc52 into espnet:master Jan 5, 2025
37 checks passed
@sw005320
Copy link
Contributor

sw005320 commented Jan 5, 2025

Thanks, @Shikhar-S!

@Shikhar-S Shikhar-S deleted the esc branch January 6, 2025 04:11
@Shikhar-S Shikhar-S mentioned this pull request Mar 2, 2025
Open
Shikhar-S pushed a commit to Shikhar-S/espnet that referenced this pull request Mar 13, 2025
@sw005320
Merge pull request espnet#5977 from Shikhar-S/esc
f65ec55 
ESC-50 classification with BEATs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ftshijt ftshijt ftshijt left review comments

@sw005320 sw005320 Awaiting requested review from sw005320

Assignees

No one assigned

Labels

ESPnet1 ESPnet2 README SED Sound Event Detection

Projects

None yet

Milestone

v.202503

Development

Successfully merging this pull request may close these issues.

3 participants

@Shikhar-S @sw005320 @ftshijt