make float8 training's force_recompute_fp8_weight_in_bwd flag do nothing #2356
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: This PR makes the `Float8LinearConfig.force_recompute_fp8_weight_in_bwd` flag do nothing and marks it for a future deprecation. Now that PyTorch Core can handle this logic automatically, we no longer need the workaround. Please see #2251 for more context. Test Plan: ``` ./test/float8/test_everything.sh ``` Reviewers: Subscribers: Tasks: Tags:
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/2356
Note: Links to docs will display an error until the docs builds have been completed. ❌ 1 New Failure, 1 Unrelated FailureAs of commit b87b329 with merge base 60c583e ( NEW FAILURE - The following job has failed:
BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
danielvegamyhre
approved these changes
Jun 11, 2025
vkuzo
added a commit
to pytorch/torchtitan
that referenced
this pull request
Jul 24, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
This was referenced Jul 24, 2025
Merged
vkuzo
added a commit
that referenced
this pull request
Jul 24, 2025
Summary: Removing code which we no longer need after #2356 . `torch.compile` now does the right thing automatically, and the relevant config has been deprecated. Also fix links in float8 training benchmark README.md to point to updated locations. Test Plan: ``` ./test/float8/test_everything.sh ``` ``` // before with-proxy TORCHTITAN_ROOT=~/local/torchtitan/ FLOAT8_RECIPE_WITH_BEST_SETTINGS="tensorwise" ./torchtitan_benchmark.sh ... Median Tokens/Second (excluding step 1): 7999.0 Max Memory Usage: 36.68 GiB // after with-proxy TORCHTITAN_ROOT=~/local/torchtitan/ FLOAT8_RECIPE_WITH_BEST_SETTINGS="tensorwise" ./torchtitan_benchmark.sh ... Median Tokens/Second (excluding step 1): 8038.5 Max Memory Usage: 36.68 GiB ``` Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 1211464 ghstack-comment-id: 3113439246 Pull Request resolved: #2594
vkuzo
added a commit
that referenced
this pull request
Jul 24, 2025
Summary: These workarounds are no longer needed after #2356 and the corresponding improvements in PyTorch core. Test Plan: torchtitan bench on llama 3 8b on 8 H100s: before rowwise Median Tokens/Second (excluding step 1): 7013.0 Max Memory Usage: 37.19 GiB gw_hp Median Tokens/Second (excluding step 1): 7232.0 Max Memory Usage: 37.13 GiB after rowwise Median Tokens/Second (excluding step 1): 6984.5 Max Memory Usage: 37.19 GiB gw_hp Median Tokens/Second (excluding step 1): 7319.5 Max Memory Usage: 37.13 GiB Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: ae11ea7 ghstack-comment-id: 3113561383 Pull Request resolved: #2595
vkuzo
added a commit
that referenced
this pull request
Jul 24, 2025
Summary: These workarounds are no longer needed after #2356 and the corresponding improvements in PyTorch core. Test Plan: torchtitan bench on llama 3 8b on 8 H100s: before rowwise Median Tokens/Second (excluding step 1): 7013.0 Max Memory Usage: 37.19 GiB gw_hp Median Tokens/Second (excluding step 1): 7232.0 Max Memory Usage: 37.13 GiB after rowwise Median Tokens/Second (excluding step 1): 6984.5 Max Memory Usage: 37.19 GiB gw_hp Median Tokens/Second (excluding step 1): 7319.5 Max Memory Usage: 37.13 GiB Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: ae11ea7 ghstack-comment-id: 3113561383 Pull Request resolved: #2595
vkuzo
added a commit
to pytorch/torchtitan
that referenced
this pull request
Jul 28, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
vkuzo
added a commit
to pytorch/torchtitan
that referenced
this pull request
Jul 28, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
Luosuu
pushed a commit
to Luosuu/torchtitan
that referenced
this pull request
Jul 28, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
bentherien
pushed a commit
to bentherien/torchtitan_
that referenced
this pull request
Aug 5, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
joellidin
pushed a commit
to one-covenant/torchtitan
that referenced
this pull request
Aug 8, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
joellidin
pushed a commit
to one-covenant/torchtitan
that referenced
this pull request
Aug 8, 2025
Summary: This flag has been deprecated in pytorch/ao#2356, deleting it from torchtitan to prepare for future deletion from torchao. Test Plan: CI Reviewers: Subscribers: Tasks: Tags:
liangel-02
pushed a commit
that referenced
this pull request
Aug 25, 2025
…ing (#2356) Summary: This PR makes the `Float8LinearConfig.force_recompute_fp8_weight_in_bwd` flag do nothing and marks it for a future deprecation. Now that PyTorch Core can handle this logic automatically, we no longer need the workaround. Please see #2251 for more context. Test Plan: ``` ./test/float8/test_everything.sh ``` Reviewers: Subscribers: Tasks: Tags:
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This PR makes the
Float8LinearConfig.force_recompute_fp8_weight_in_bwdflag do nothing and marks it for a future deprecation.Now that PyTorch Core can handle this logic automatically, we no longer need the workaround. Please
see #2251 for more context.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags: