Commit 2386a91
Distributed optimizer infrastructure for FP8 parameters (#1723)
* Add distopt support for param syncs with non-floating-point dtypes
Signed-off-by: Tim Moon <[email protected]>
* Update apex/contrib/optimizers/distributed_fused_adam.py
Co-authored-by: Masaki Kozuki <[email protected]>
---------
Signed-off-by: Tim Moon <[email protected]>
Co-authored-by: Masaki Kozuki <[email protected]>1 parent 6ff4548 commit 2386a91
2 files changed
Lines changed: 280 additions & 83 deletions
0 commit comments