Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Mar 29, 2024

• fix xentropy bf16 bug

  zyeric

  committed
  0e4a0b6

  Copy full SHA for 0e4a0b6

Commits on Mar 12, 2024

• Update test_fused_softmax.py (#1782)

  eqy

  authored
  810ffae

  Copy full SHA for 810ffae

Commits on Feb 8, 2024

• Support scaled optimizer state in distributed Adam optimizer (#1771)

  Show description for b496d85
  timmoon10

  authored
  b496d85

  Copy full SHA for b496d85

Commits on Feb 7, 2024

• Add GPUDirect Storage (#1774)

  Show description for 5b67cd5
  Aidyn-A

  authored
  5b67cd5

  Copy full SHA for 5b67cd5

• Skip the p2p test on single GPU platforms (#1775)

  nWEIdia

  authored
  7e239f7

  Copy full SHA for 7e239f7

Commits on Jan 25, 2024

• Update test_adam.py (#1772)

  eqy

  authored
  141bbf1

  Copy full SHA for 141bbf1

Commits on Jan 18, 2024

• Update test_bottleneck_module.py - Skip BottleNeck Peer Memory Test if Not Supported (#1769)

  Show description for 6c8f384
  nWEIdia

  authored
  6c8f384

  Copy full SHA for 6c8f384

Commits on Jan 12, 2024

• Update test_transducer_joint.py (#1767)

  Show description for 48c4894
  nWEIdia

  authored
  48c4894

  Copy full SHA for 48c4894

• Fused RoPE for `thd` format (#1756)

  Show description for f058162
  yaox12

  authored
  f058162

  Copy full SHA for f058162

Commits on Jan 10, 2024

• Increase tolerance to workaround tolerance issues on A100 (#1766)

  Show description for e9789cc
  nWEIdia

  authored
  e9789cc

  Copy full SHA for e9789cc

Commits on Jan 5, 2024

• 64-bit indexing Adam (#1765)

  Show description for 87c4deb
  eqy

  authored
  87c4deb

  Copy full SHA for 87c4deb

Commits on Jan 1, 2024

• Make fused layer norm functions backward-compatible (#1760)

  Show description for c07a4cf
  timmoon10

  authored
  c07a4cf

  Copy full SHA for c07a4cf

Commits on Dec 15, 2023

• [contrib] Improve FusedAdamSWA interface and add unit tests (#1759)

  Show description for ccffcc4
  

  lirundong

  and

  crcrpar

  authored
  ccffcc4

  Copy full SHA for ccffcc4

• add async copy for openfold swa (#1758)

  Show description for 5d89c04
  azazhu

  authored
  5d89c04

  Copy full SHA for 5d89c04

Commits on Nov 29, 2023

• [FusedRoPE] Fuse type conversion and cos/sin (#1752)

  Show description for 37d83fc
  yaox12

  authored
  37d83fc

  Copy full SHA for 37d83fc

Commits on Nov 28, 2023

• Use recommended PyTorch methods to silence warnings (#1754)

  Show description for 7548f68
  deepakn94

  authored
  7548f68

  Copy full SHA for 7548f68

Commits on Nov 23, 2023

• Avoid `.contiguous()` in fused RoPE (#1751)

  Show description for dc5fa38
  yaox12

  authored
  dc5fa38

  Copy full SHA for dc5fa38

Commits on Nov 20, 2023

• Distributed optimizer support for contiguous param buffer with FP8 params (#1749)

  Show description for a2f6683
  timmoon10

  authored
  a2f6683

  Copy full SHA for a2f6683

Commits on Nov 16, 2023

• fix a bug in fused rope (#1750)

  Show description for fd4ae7d
  yaox12

  authored
  fd4ae7d

  Copy full SHA for fd4ae7d

Commits on Nov 14, 2023

• A fused `apply_rotary_pos_emb` implementation for Megatron-Core (#1746)

  Show description for 08f7402
  yaox12

  authored
  08f7402

  Copy full SHA for 08f7402

Commits on Nov 10, 2023

• Misc Changes (#1747)

  Show description for 97e38d6
  nWEIdia

  authored
  97e38d6

  Copy full SHA for 97e38d6

Commits on Nov 8, 2023

• Add precision comments.

  

  alpha0422

  authored and

  crcrpar

  committed
  acd8950

  Copy full SHA for acd8950

• Update apex/contrib/csrc/group_norm/group_norm_nhwc.cpp

  Show description for 5a6529f
  

  alpha0422

  and

  crcrpar

  committed
  5a6529f

  Copy full SHA for 5a6529f

• Port changes until a9396c36.

  Show description for 46befc2
  

  2 peopleauthored and

  crcrpar

  committed
  46befc2

  Copy full SHA for 46befc2

• Add explicit dtype checks.

  

  alpha0422

  authored and

  crcrpar

  committed
  72ae42a

  Copy full SHA for 72ae42a

• Port changes until 108e6739.

  Show description for 464b8f1
  

  3 peopleauthored and

  crcrpar

  committed
  464b8f1

  Copy full SHA for 464b8f1

Commits on Oct 19, 2023

• Specify `rtol` as well as `atol` in test_fmha.py (#1743)

  eqy

  authored
  9fc94b7

  Copy full SHA for 9fc94b7

• Loop through all available engines for cuDNN heuristics search (#1740)

  Show description for ddd9b3f
  minitu

  authored
  ddd9b3f

  Copy full SHA for ddd9b3f

Commits on Oct 17, 2023

• Use `torch.testing.assert_close` for better signals (#1742)

  eqy

  authored
  19cc873

  Copy full SHA for 19cc873

Commits on Oct 13, 2023

• add test for openfold triton mha kernel (#1739)

  Show description for 5afc6a0
  azazhu

  authored
  5afc6a0

  Copy full SHA for 5afc6a0

Commits on Oct 3, 2023

• Add openfold triton code (#1734)

  Show description for 58acf96
  arnowaczynski

  authored
  58acf96

  Copy full SHA for 58acf96

Commits on Sep 30, 2023

• Add hysteresis support for AMP gradient scale update (#1733)

  Show description for 6a77872
  

  4 peopleauthored
  6a77872

  Copy full SHA for 6a77872

Commits on Sep 29, 2023

• Distributed optimizer infrastructure for FP8 parameters (#1723)

  Show description for 2386a91
  

  timmoon10

  and

  crcrpar

  authored
  2386a91

  Copy full SHA for 2386a91

Commits on Sep 28, 2023

• Massively reduce LayerNorm/RMSNorm GPU memory usage in modern networks by tricking torch autograd (#1715)

  Show description for 6ff4548
  RuiWang1998

  authored
  6ff4548

  Copy full SHA for 6ff4548

Commits on Sep 19, 2023

• Add multi_tensor_unscale_l2norm_cuda (#1727)

  Show description for 741bdf5
  minitu

  authored
  741bdf5

  Copy full SHA for 741bdf5