While reading your new technical report, I had a question about the LM section.
Specifically, on page 4, you write:
“Crucially, the diffusion process is trained to predict only the acoustic VAE za,i. Concurrently,
when the LLM generates the conditioning hidden state hi for the diffusion head, it is also tasked
with predicting whether the current speech segment should conclude, effectively determining if a
diffusion termination token (e.g., <E>) should be emitted at this step.”
For this part, are you using CE loss for the token, or did you add a separate binary loss for ?
Thank you.
While reading your new technical report, I had a question about the LM section.
Specifically, on page 4, you write:
For this part, are you using CE loss for the token, or did you add a separate binary loss for ?
Thank you.