Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on May 18, 2026

• Fix E721 type comparisons (#2003)

  LeSingh1

  authored
  33254f2

  Copy full SHA for 33254f2

Commits on Nov 28, 2025

• Enable Ruff pre-commit hooks (#1957)

  Show description for 16fadfe
  

  3 peopleauthored
  16fadfe

  Copy full SHA for 16fadfe

Commits on Apr 10, 2025

• Pipeline `reduce-scatter` and `all-reduce`. (#1895)

  alpha0422

  authored
  b9d758c

  Copy full SHA for b9d758c

Commits on Apr 9, 2025

• Fix `DistributedFusedAdam` for grad dtype != param dtype (#1893)

  yuantailing

  authored
  2eb64da

  Copy full SHA for 2eb64da

Commits on Feb 26, 2025

• [nccl_allocator] Adds helper API to create pool (#1877)

  syed-ahmed

  authored
  312acb4

  Copy full SHA for 312acb4

Commits on Oct 17, 2024

• Avoid NCCL collective coalescing in distopt when not needed (#1847)

  Show description for 6102d2c
  timmoon10

  authored
  6102d2c

  Copy full SHA for 6102d2c

Commits on Aug 30, 2024

• Add Unittest For Distributed Adam With CUDA Graph (#1836)

  Show description for b7a4acc
  

  alpha0422

  and

  crcrpar

  authored
  b7a4acc

  Copy full SHA for b7a4acc

• Enhance Distributed Fused Adam (#1832)

  Show description for 7d5ecf1
  

  alpha0422

  and

  timmoon10

  authored
  7d5ecf1

  Copy full SHA for 7d5ecf1

Commits on Jul 24, 2024

• Do not monkey-patch class methods when registering distopt pre-forward hooks (#1820)

  Show description for f9f19d5
  timmoon10

  authored
  f9f19d5

  Copy full SHA for f9f19d5

Commits on Apr 24, 2024

• NCCL userbuffer for DP RS in DistOpt (#1797)

  Show description for f3f0492
  

  WanZzzzzz

  and

  Qiyu Wan

  authored
  f3f0492

  Copy full SHA for f3f0492

Commits on Apr 19, 2024

• move to correct device for v1 state (#1783)

  acphile

  authored
  c5f6b79

  Copy full SHA for c5f6b79

Commits on Feb 8, 2024

• Support scaled optimizer state in distributed Adam optimizer (#1771)

  Show description for b496d85
  timmoon10

  authored
  b496d85

  Copy full SHA for b496d85

Commits on Nov 20, 2023

• Distributed optimizer support for contiguous param buffer with FP8 params (#1749)

  Show description for a2f6683
  timmoon10

  authored
  a2f6683

  Copy full SHA for a2f6683

Commits on Sep 29, 2023

• Distributed optimizer infrastructure for FP8 parameters (#1723)

  Show description for 2386a91
  

  timmoon10

  and

  crcrpar

  authored
  2386a91

  Copy full SHA for 2386a91

Commits on Sep 6, 2023

• Distributed optimizer support for multiple dtypes (#1721)

  Show description for 52e18c8
  timmoon10

  authored
  52e18c8

  Copy full SHA for 52e18c8

Commits on Aug 29, 2023

• Include format version in distopt checkpoints (#1716)

  Show description for 5b04afc
  

  timmoon10

  and

  crcrpar

  authored
  5b04afc

  Copy full SHA for 5b04afc

Commits on Aug 28, 2023

• Add the warning of distributed_fused_adam low bucket usage (#1714)

  Show description for 1d01e5c
  

  shjwudp

  and

  timmoon10

  authored
  1d01e5c

  Copy full SHA for 1d01e5c

Commits on Aug 19, 2023

• DP-independent checkpoint format for distributed Adam optimizer (#1704)

  Show description for 7995de1
  timmoon10

  authored
  7995de1

  Copy full SHA for 7995de1

Commits on Jul 31, 2023

• Add type hints to distributed Adam optimizer (#1699)

  Show description for 2958e06
  timmoon10

  authored
  2958e06

  Copy full SHA for 2958e06

Commits on May 13, 2023

• Update distributed optimizer with new coalescing manager API in PyTorch (#1663)

  Show description for 8b7a1ff
  

  timmoon10

  and

  crcrpar

  authored
  8b7a1ff

  Copy full SHA for 8b7a1ff

Commits on Mar 23, 2023

• Overlap first param sync with optimizer step in distributed Adam (#1621)

  Show description for 7150e20
  timmoon10

  authored
  7150e20

  Copy full SHA for 7150e20

Commits on Mar 17, 2023

• Align distopt communication in interleaved pipeline parallelism (#1611)

  Show description for 27de66c
  

  timmoon10

  and

  crcrpar

  authored
  27de66c

  Copy full SHA for 27de66c

Commits on Feb 25, 2023

• Decouple optimizer state and grad dtypes in distributed Adam optimizer (#1575)

  Show description for 03c9d80
  timmoon10

  authored
  03c9d80

  Copy full SHA for 03c9d80

Commits on Feb 7, 2023

• Distributed optimizer overlaps forward compute and param all-gather (#1559)

  Show description for c0a0b0f
  timmoon10

  authored
  c0a0b0f

  Copy full SHA for c0a0b0f

Commits on Jan 20, 2023

• Optimizing distributed Adam when running with one work queue (#1551)

  Show description for 75f401e
  timmoon10

  authored
  75f401e

  Copy full SHA for 75f401e

Commits on Dec 9, 2022

• change param of list of tensor to tensor for _make_nccl_premul_sum (#1542)

  Show description for 91fcaaf
  crcrpar

  authored
  91fcaaf

  Copy full SHA for 91fcaaf

Commits on Nov 21, 2022

• [contrib] fix `reduce_scatter_tensor` fallouts in DistributedFusedLamb (#1536)

  Show description for c216175
  

  crcrpar

  and

  mikolajblaz

  authored
  c216175

  Copy full SHA for c216175

Commits on Nov 17, 2022

• Perform dist Adam communication in side stream (#1528)

  Show description for d79f69a
  timmoon10

  authored
  d79f69a

  Copy full SHA for d79f69a

Commits on Oct 26, 2022

• Use all_gather_into_tensor and reduce_scatter_tensor (#1513)

  Show description for f680fab
  Aidyn-A

  authored
  f680fab

  Copy full SHA for f680fab

Commits on Oct 25, 2022

• Fix bug with grad clipping and distributed Adam (#1512)

  Show description for da9f5ae
  timmoon10

  authored
  da9f5ae

  Copy full SHA for da9f5ae

Commits on Oct 2, 2022

• Replace usage of internal with public APIs (#1496)

  Show description for 6816ef6
  kumpera

  authored
  6816ef6

  Copy full SHA for 6816ef6

Commits on Sep 20, 2022

• [contrib][DistributedFusedAdam] Support overlapped grad sync with Megatron pipeline parallelism (#1475)

  Show description for 2b0e837
  timmoon10

  authored
  2b0e837

  Copy full SHA for 2b0e837

Commits on Aug 25, 2022

• [contrib] Guard for `torch.distributed._make_nccl_premul_sum` (#1465)

  Show description for 7e90f95
  crcrpar

  authored
  7e90f95

  Copy full SHA for 7e90f95

• Optimize distributed Adam kernels and implementation (#1463)

  Show description for 6f7d8ac
  timmoon10

  authored
  6f7d8ac

  Copy full SHA for 6f7d8ac

Commits on Aug 10, 2022

• Store optimizer state_dict in uint8 tensor instead of bytearray (#1446)

  Show description for a0f5f3a
  timmoon10

  authored
  a0f5f3a

  Copy full SHA for a0f5f3a