Enable XPU distributed test for PT2.8 #149916
Conversation
This reverts commit f5cbd50.
This reverts commit c791db9.
This reverts commit d0d8271.
Signed-off-by: Cheng Penghui <[email protected]>
…into distributed_2.8
Signed-off-by: Cheng Penghui <[email protected]>
Signed-off-by: Cheng Penghui <[email protected]>
…into distributed_2.8
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/149916
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ No FailuresAs of commit 0e7a7b6 with merge base a09a3f4 ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Signed-off-by: Cheng Penghui <[email protected]>
| @@ -19,6 +19,7 @@ | |||
| TransformerBlock, | |||
| ) | |||
|
|
|||
| device_type = torch.accelerator.current_accelerator().type | |||
There was a problem hiding this comment.
Here will raise an Error if the current accelerator returns None.
| default_stream = torch.cuda.current_stream() | ||
| stream = torch.cuda.Stream() | ||
| default_stream = torch.accelerator.current_stream() | ||
| stream = torch.xpu.Stream() if device_type == "xpu" else torch.cuda.Stream() |
There was a problem hiding this comment.
| stream = torch.xpu.Stream() if device_type == "xpu" else torch.cuda.Stream() | |
| stream = torch.Stream() |
|
@daisyden This PR involves a significant code change, so I prefer to mark it as a draft until it's CI-ready and the internal review comments have been addressed. The main goal of this PR is to generalize the unit tests to be device-agnostic and enable XPU support on top of it. Would it be possible to split this PR into two separate ones: one focused on the generalization, and the other aimed at enabling XPU? |
Signed-off-by: Cheng Penghui <[email protected]>
…equires_nccl_or and requires_nccl_version_or to replace requires_nccl and requires_nccl_version when xccl test is enabled on a test
Signed-off-by: Cheng Penghui <[email protected]>
Signed-off-by: Cheng Penghui <[email protected]>
Signed-off-by: Cheng Penghui <[email protected]>
…into distributed_2.8
Signed-off-by: Cheng, Penghui <[email protected]>
…into distributed_2.8
…into distributed_2.8
Signed-off-by: Cheng Penghui <[email protected]>
Signed-off-by: Cheng, Penghui <[email protected]>
Signed-off-by: Cheng, Penghui <[email protected]>
|
To add the ciflow label This helps ensure we don't trigger CI on this PR until it is actually authorized to do so. Please ping one of the reviewers if you do not have access to approve and run workflows. |
Signed-off-by: Cheng, Penghui <[email protected]>
Signed-off-by: Cheng, Penghui <[email protected]>
Fixes #ISSUE_NUMBER
cc @H-Huang @awgu @wanchaol @fegin @fduwjj @wz337 @wconstab @d4l3k @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @ipiszy @chenyang78 @kadeng @muchulee8 @amjames @chauhang @aakhundov @kwen2501 @c-p-i-o