Force upsample to be float32 #121324
Force upsample to be float32 #121324
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/121324
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit 04a831d with merge base 47330ca ( FLAKY - The following job failed but was likely due to flakiness present on trunk:
BROKEN TRUNK - The following job failed but was present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot merge |
Merge failedReason: This PR needs a If not, please add the To add a label, you can comment to pytorchbot, for example For more information, see Details for Dev Infra teamRaised by workflow job |
|
About the user facing doc I don't see that we had upsample in the list right? Is this probably cause that list is not auto-generated and it is out of sync? |
|
Green lights here. |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Merge failedReason: 12 mandatory check(s) failed. The first few are:
Dig deeper by viewing the failures on hud |
|
Are these errors related to this PR? |
|
/cc @nWEIdia |
|
|
Thanks @albanD the question was general not only about the refactoring as it is not clear if these ops are still impacted by the amp float32 enforcing machinery when lowered/compiled. |
|
@albanD It was adapted to the mentioned refactoring. |
Atomic add changes were properly removed
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
|
The merge job was canceled. If you believe this is a mistake, then you can re trigger it through pytorch-bot. |
|
@pytorchbot merge |
Merge startedYour change will be merged once all checks pass (ETA 0-4 Hours). Learn more about merging in the wiki. Questions? Feedback? Please reach out to the PyTorch DevX Team |
Fixes pytorch#121072 Pull Request resolved: pytorch#121324 Approved by: https://github.com/albanD
|
This diff regresses torchbench pytorch_unet by 10%: pytorch/benchmark#2247 |
|
Regressed in what sense? Speed or numerical? |
It regressed in latency ~10% and peak gpu memory usage ~40%. In pytorch/benchmark CI we only run eager mode (PT1) so there is no numerical metric. |
|
I guess this is expected behavior given that we now upcast this op. cc @ezyang is that a regression that is problematic? |
|
This is an explicit correctness/speed tradeoff here. @bhack, is the memory usage regression you were expecting to see in this case? |
|
A regression is expected for sure as we are working at
Cause the gradient seems better at lower precision with the compiled code. |
Fixes #121072 Pull Request resolved: #121324 Approved by: https://github.com/albanD
Fixes #121072
BC-breaking note:
This is not technically bc-breaking any behavior but will lead to an expected significant performance change for amp + deterministic.
cc @zou3519 any way we can tell people how to register a fallthrough for that key to recover the old behavior if they want?
cc @mcarilli @ptrblck @leslie-fang-intel @jgong5 @voznesenskym @penguinwu @EikanWang @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @peterbell10 @ipiszy @yf225 @chenyang78 @kadeng @muchulee8 @aakhundov @ColinPeppler @amjames @desertfire @chauhang