[Inductor] Skip nonfinite preservation for safe scaled softmax#186211
[Inductor] Skip nonfinite preservation for safe scaled softmax#186211Yuxingwang-intel wants to merge 1 commit into
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/186211
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
|
The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:
Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows. |
1 similar comment
|
The following ciflow label(s) have been added but CI has not been triggered yet because the workflows are awaiting approval:
Once a maintainer approves the workflows (scroll to the bottom of the PR page), the corresponding CI jobs will be triggered automatically. Please ping one of the reviewers if you do not have access to approve and run workflows. |
Skip the scaled softmax nonfinite-semantics preservation logic for static safe scale factors.
PR #184046 added logic in Inductor to preserve the original scaled subtraction behavior for nonfinite scaled-softmax inputs, so that Inductor does not turn eager NaNs into finite softmax results. However, for static positive scales <= 1, multiplying finite inputs by the scale will not introduce overflow. In those cases, the additional preservation logic is unnecessary overhead and can also block downstream fusion patterns or cause performance regressions on benchmarks.
This PR keeps the existing nonfinite-semantics preserving path for unsafe or dynamic scales but directly returns the stable scaled-softmax form for compile-time safe scales.
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @kadeng @muchulee8 @amjames @chauhang @aakhundov @coconutruben @jataylo