cc @Cyrilvallez for generation!

Could you please provide a reproducer? I don't see how all logits could end up at -inf...

A full -inf row can happen if a previous logits processor masks everything for a step (misconfigured constraints/custom callback). A minimal reproducer is:

import torch
from transformers.generation.logits_process import SuppressTokensLogitsProcessor, EtaLogitsWarper

input_ids = torch.tensor([[0]])
scores = torch.tensor([[5.0, 1.0, -1.0, 0.2]])

# Previous processor masks all vocab entries
masked = SuppressTokensLogitsProcessor([0, 1, 2, 3])(input_ids, scores)
print(masked)
# tensor([[-inf, -inf, -inf, -inf]])

# This PR keeps eta stable on this input
out = EtaLogitsWarper(epsilon=1e-3, device="cpu")(input_ids, masked)
print(out, torch.isnan(out).any())
# tensor([[-inf, -inf, -inf, -inf]]) tensor(False)

Before this PR, the entropy path used in EtaLogitsWarper would go through Categorical(logits=scores).entropy(), which raises here because softmax([-inf, ...]) becomes NaN.

So this patch is mainly a robustness guard: preserve the fully masked state and avoid NaNs/crash in eta when upstream processors produce that edge case.

Humm, I think it actually makes more sense to keep the error and crash on such cases, rather than silently fail when something is wrong. A full row should never be all -inf, and if it's the case as you said something is really wrong and it's better to crash.

yup, thanks for the feedback.
I updated the PR accordingly: it now fails fast instead of silently recovering.
n EtaLogitsWarper, if any row is fully masked (all -inf), we now raise a clear ValueError immediately, the regression test was updated to assert that explicit error behavior.
So this now keeps the crash semantics you suggested, but with a more direct error message indicating that an upstream processor masked every token.

Small follow-up pushed on top: moved the fully-masked-row guard in EtaLogitsWarper to run before softmax, so we fail fast without doing any NaN-producing computation first.

Hi @Cyrilvallez , just a quick follow-up on this PR. Is there anything still blocking it, or is it simply waiting for review? Happy to make any additional changes if needed.

My main issue with the PR is that we perform runtime value checks that should not be necessary. I.e. we never do it in any other Processor, because there is no reason in general to end up with fully masked rows...

My main issue with the PR is that we perform runtime value checks that should not be necessary. I.e. we never do it in any other Processor, because there is no reason in general to end up with fully masked rows...

That makes sense , I’ll close this PR. Thanks for the review and clarification.