Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Get forced alignments from CTC model#6248

Merged
sw005320 merged 22 commits intoespnet:masterfrom
Shikhar-S:powsm
Oct 7, 2025
Merged

Get forced alignments from CTC model#6248
sw005320 merged 22 commits intoespnet:masterfrom
Shikhar-S:powsm

Conversation

@Shikhar-S
Copy link
Contributor

@Shikhar-S Shikhar-S commented Sep 23, 2025

What did you change?

Add forced alignment outputs based on CTC posteriors for the S2T model.

Why did you make this change?

To get forced alignments from any CTC models in ESPnet.

Is your PR small enough?

Yes

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Sep 23, 2025
@mergify mergify bot added the ESPnet2 label Sep 23, 2025
@dosubot dosubot bot added the Force alignment including CTC segmentation label Sep 23, 2025
gemini-code-assist[bot]
gemini-code-assist bot reviewed Sep 23, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces forced alignment capabilities for CTC models. The implementation looks solid and is accompanied by relevant tests. I've identified two high-severity issues in the new code. One is a potential correctness issue in espnet2/asr/ctc.py where a precondition for an external function is not validated, which could lead to incorrect results or cryptic errors. The other is in espnet2/s2t/espnet_model.py regarding an in-place modification of an input tensor, which is a dangerous side-effect and can cause subtle bugs. I've provided suggestions to address both.

pre-commit-ci bot and others added 3 commits September 23, 2025 00:24
@pre-commit-ci
[pre-commit.ci] auto fixes from pre-commit.com hooks
7defb8e 
for more information, see https://pre-commit.ci
@Shikhar-S @gemini-code-assist
Update espnet2/asr/ctc.py
4d7693f 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@Shikhar-S @gemini-code-assist
Update espnet2/s2t/espnet_model.py
265913a 
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
@Fhrozen Fhrozen added this to the v.202512 milestone Sep 23, 2025
@sw005320
Copy link
Contributor

sw005320 commented Sep 23, 2025

Thanks!
This is useful.
How (and when) would it be enabled?
Any example?

@sw005320
Copy link
Contributor

sw005320 commented Sep 23, 2025

Also, please fix the CI errors

@dosubot dosubot bot added size:XL This PR changes 500-999 lines, ignoring generated files. and removed size:L This PR changes 100-499 lines, ignoring generated files. labels Oct 2, 2025
@mergify mergify bot added the CI Travis, Circle CI, etc label Oct 2, 2025
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels Oct 2, 2025
@Shikhar-S
Copy link
Contributor Author

Shikhar-S commented Oct 2, 2025 •
edited
Loading

Thanks! This is useful. How (and when) would it be enabled? Any example?

Thanks for review. Added an example here - egs2/ipapack_plus/s2t1/force_align.py
It can only be used during inference in current implementation.

@codecov
Copy link

codecov bot commented Oct 2, 2025 •
edited
Loading

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 56.78%. Comparing base (1d0c1db) to head (a9a3476).
⚠️ Report is 26 commits behind head on master.

Additional details and impacted files 
@@           Coverage Diff           @@
##           master    #6248   +/-   ##
=======================================
  Coverage   56.77%   56.78%           
=======================================
  Files         889      889           
  Lines       84340    84361   +21     
=======================================
+ Hits        47882    47902   +20     
- Misses      36458    36459    +1
Flag Coverage Δ
test_integration_espnet2 46.80% <9.52%> (-0.02%) ⬇️
test_integration_espnetez 36.92% <9.52%> (-0.02%) ⬇️
test_python_espnet2 51.20% <100.00%> (+0.01%) ⬆️
test_python_espnetez 12.81% <9.52%> (-0.01%) ⬇️
test_utils 18.77% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Shikhar-S
Copy link
Contributor Author

Shikhar-S commented Oct 7, 2025

Hi, this is ready to be merged.
@sw005320

@sw005320 sw005320 merged commit 1778902 into espnet:master Oct 7, 2025
32 checks passed
@sw005320
Copy link
Contributor

sw005320 commented Oct 7, 2025

OK, thanks!

@Fhrozen Fhrozen modified the milestones: v.202512, v.202511 Nov 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

CI Travis, Circle CI, etc ESPnet2 Force alignment including CTC segmentation size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Milestone

v.202511

Development

Successfully merging this pull request may close these issues.

3 participants

@Shikhar-S @sw005320 @Fhrozen