Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Nov 28, 2025

• Enable Ruff pre-commit hooks (#1957)

  Show description for 16fadfe
  

  3 peopleauthored
  16fadfe

  Copy full SHA for 16fadfe

Commits on Apr 10, 2025

• Pipeline `reduce-scatter` and `all-reduce`. (#1895)

  alpha0422

  authored
  b9d758c

  Copy full SHA for b9d758c

Commits on Apr 9, 2025

• Fix `DistributedFusedAdam` for grad dtype != param dtype (#1893)

  yuantailing

  authored
  2eb64da

  Copy full SHA for 2eb64da

Commits on Feb 26, 2025

• [nccl_allocator] Adds helper API to create pool (#1877)

  syed-ahmed

  authored
  312acb4

  Copy full SHA for 312acb4

Commits on Oct 17, 2024

• Avoid NCCL collective coalescing in distopt when not needed (#1847)

  Show description for 6102d2c
  timmoon10

  authored
  6102d2c

  Copy full SHA for 6102d2c

Commits on Aug 30, 2024

• Add Unittest For Distributed Adam With CUDA Graph (#1836)

  Show description for b7a4acc
  

  alpha0422

  and

  crcrpar

  authored
  b7a4acc

  Copy full SHA for b7a4acc

• Enhance Distributed Fused Adam (#1832)

  Show description for 7d5ecf1
  

  alpha0422

  and

  timmoon10

  authored
  7d5ecf1

  Copy full SHA for 7d5ecf1

Commits on Jul 24, 2024

• Do not monkey-patch class methods when registering distopt pre-forward hooks (#1820)

  Show description for f9f19d5
  timmoon10

  authored
  f9f19d5

  Copy full SHA for f9f19d5

Commits on Apr 24, 2024

• NCCL userbuffer for DP RS in DistOpt (#1797)

  Show description for f3f0492
  

  WanZzzzzz

  and

  Qiyu Wan

  authored
  f3f0492

  Copy full SHA for f3f0492

Commits on Apr 19, 2024

• move to correct device for v1 state (#1783)

  acphile

  authored
  c5f6b79

  Copy full SHA for c5f6b79

Commits on Feb 8, 2024

• Support scaled optimizer state in distributed Adam optimizer (#1771)

  Show description for b496d85
  timmoon10

  authored
  b496d85

  Copy full SHA for b496d85

Commits on Nov 20, 2023

• Distributed optimizer support for contiguous param buffer with FP8 params (#1749)

  Show description for a2f6683
  timmoon10

  authored
  a2f6683

  Copy full SHA for a2f6683

Commits on Sep 29, 2023

• Distributed optimizer infrastructure for FP8 parameters (#1723)

  Show description for 2386a91
  

  timmoon10

  and

  crcrpar

  authored
  2386a91

  Copy full SHA for 2386a91

Commits on Sep 6, 2023

• Distributed optimizer support for multiple dtypes (#1721)

  Show description for 52e18c8
  timmoon10

  authored
  52e18c8

  Copy full SHA for 52e18c8

Commits on Aug 29, 2023

• Include format version in distopt checkpoints (#1716)

  Show description for 5b04afc
  

  timmoon10

  and

  crcrpar

  authored
  5b04afc

  Copy full SHA for 5b04afc

Commits on Aug 28, 2023

• Add the warning of distributed_fused_adam low bucket usage (#1714)

  Show description for 1d01e5c
  

  shjwudp

  and

  timmoon10

  authored
  1d01e5c

  Copy full SHA for 1d01e5c

Commits on Aug 19, 2023

• DP-independent checkpoint format for distributed Adam optimizer (#1704)

  Show description for 7995de1
  timmoon10

  authored
  7995de1

  Copy full SHA for 7995de1

Commits on Jul 31, 2023

• Add type hints to distributed Adam optimizer (#1699)

  Show description for 2958e06
  timmoon10

  authored
  2958e06

  Copy full SHA for 2958e06

Commits on May 13, 2023

• Update distributed optimizer with new coalescing manager API in PyTorch (#1663)

  Show description for 8b7a1ff
  

  timmoon10

  and

  crcrpar

  authored
  8b7a1ff

  Copy full SHA for 8b7a1ff

Commits on Mar 23, 2023

• Overlap first param sync with optimizer step in distributed Adam (#1621)

  Show description for 7150e20
  timmoon10

  authored
  7150e20

  Copy full SHA for 7150e20

Commits on Mar 17, 2023

• Align distopt communication in interleaved pipeline parallelism (#1611)

  Show description for 27de66c
  

  timmoon10

  and

  crcrpar

  authored
  27de66c

  Copy full SHA for 27de66c

Commits on Feb 25, 2023

• Decouple optimizer state and grad dtypes in distributed Adam optimizer (#1575)

  Show description for 03c9d80
  timmoon10

  authored
  03c9d80

  Copy full SHA for 03c9d80

Commits on Feb 7, 2023

• Distributed optimizer overlaps forward compute and param all-gather (#1559)

  Show description for c0a0b0f
  timmoon10

  authored
  c0a0b0f

  Copy full SHA for c0a0b0f

Commits on Jan 20, 2023

• Optimizing distributed Adam when running with one work queue (#1551)

  Show description for 75f401e
  timmoon10

  authored
  75f401e

  Copy full SHA for 75f401e

Commits on Nov 17, 2022

• Perform dist Adam communication in side stream (#1528)

  Show description for d79f69a
  timmoon10

  authored
  d79f69a

  Copy full SHA for d79f69a

Commits on Oct 25, 2022

• Fix bug with grad clipping and distributed Adam (#1512)

  Show description for da9f5ae
  timmoon10

  authored
  da9f5ae

  Copy full SHA for da9f5ae

Commits on Oct 2, 2022

• Replace usage of internal with public APIs (#1496)

  Show description for 6816ef6
  kumpera

  authored
  6816ef6

  Copy full SHA for 6816ef6

Commits on Sep 20, 2022

• [contrib][DistributedFusedAdam] Support overlapped grad sync with Megatron pipeline parallelism (#1475)

  Show description for 2b0e837
  timmoon10

  authored
  2b0e837

  Copy full SHA for 2b0e837

Commits on Aug 25, 2022

• Optimize distributed Adam kernels and implementation (#1463)

  Show description for 6f7d8ac
  timmoon10

  authored
  6f7d8ac

  Copy full SHA for 6f7d8ac

Commits on Aug 10, 2022

• Store optimizer state_dict in uint8 tensor instead of bytearray (#1446)

  Show description for a0f5f3a
  timmoon10

  authored
  a0f5f3a

  Copy full SHA for a0f5f3a

Commits on Jul 26, 2022

• Improvements in distributed Adam optimizer for Megatron (#1432)

  Show description for 2e025ab
  timmoon10

  authored
  2e025ab

  Copy full SHA for 2e025ab

Commits on Jul 5, 2022

• Add features to distributed Adam for Megatron support (#1414)

  Show description for cd49973
  timmoon10

  authored
  cd49973

  Copy full SHA for cd49973

Commits on Jun 23, 2022

• Move distributed Adam unit test to contrib dir (#1406)

  Show description for 57f890a
  timmoon10

  authored
  57f890a

  Copy full SHA for 57f890a

Commits on Jun 14, 2022

• Update documentation to reflect DistributedFusedAdam uses AdamW

  Show description for 846f7f8
  timmoon10

  committed
  846f7f8

  Copy full SHA for 846f7f8

Commits on Jun 13, 2022

• Add ZeRO-2 support in DistributedFusedAdam

  timmoon10

  committed
  6e41291

  Copy full SHA for 6e41291