Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Dec 15, 2023

• [contrib] Improve FusedAdamSWA interface and add unit tests (#1759)

  Show description for ccffcc4
  

  lirundong

  and

  crcrpar

  authored
  ccffcc4

  Copy full SHA for ccffcc4

• add async copy for openfold swa (#1758)

  Show description for 5d89c04
  azazhu

  authored
  5d89c04

  Copy full SHA for 5d89c04

Commits on Nov 29, 2023

• [FusedRoPE] Fuse type conversion and cos/sin (#1752)

  Show description for 37d83fc
  yaox12

  authored
  37d83fc

  Copy full SHA for 37d83fc

Commits on Nov 28, 2023

• Use recommended PyTorch methods to silence warnings (#1754)

  Show description for 7548f68
  deepakn94

  authored
  7548f68

  Copy full SHA for 7548f68

Commits on Nov 23, 2023

• Avoid `.contiguous()` in fused RoPE (#1751)

  Show description for dc5fa38
  yaox12

  authored
  dc5fa38

  Copy full SHA for dc5fa38

Commits on Nov 20, 2023

• Distributed optimizer support for contiguous param buffer with FP8 params (#1749)

  Show description for a2f6683
  timmoon10

  authored
  a2f6683

  Copy full SHA for a2f6683

Commits on Nov 16, 2023

• fix a bug in fused rope (#1750)

  Show description for fd4ae7d
  yaox12

  authored
  fd4ae7d

  Copy full SHA for fd4ae7d

Commits on Nov 14, 2023

• A fused `apply_rotary_pos_emb` implementation for Megatron-Core (#1746)

  Show description for 08f7402
  yaox12

  authored
  08f7402

  Copy full SHA for 08f7402

Commits on Nov 10, 2023

• Misc Changes (#1747)

  Show description for 97e38d6
  nWEIdia

  authored
  97e38d6

  Copy full SHA for 97e38d6

Commits on Nov 8, 2023

• Add precision comments.

  

  alpha0422

  authored and

  crcrpar

  committed
  acd8950

  Copy full SHA for acd8950

• Update apex/contrib/csrc/group_norm/group_norm_nhwc.cpp

  Show description for 5a6529f
  

  alpha0422

  and

  crcrpar

  committed
  5a6529f

  Copy full SHA for 5a6529f

• Port changes until a9396c36.

  Show description for 46befc2
  

  2 peopleauthored and

  crcrpar

  committed
  46befc2

  Copy full SHA for 46befc2

• Add explicit dtype checks.

  

  alpha0422

  authored and

  crcrpar

  committed
  72ae42a

  Copy full SHA for 72ae42a

• Port changes until 108e6739.

  Show description for 464b8f1
  

  3 peopleauthored and

  crcrpar

  committed
  464b8f1

  Copy full SHA for 464b8f1

Commits on Oct 19, 2023

• Specify `rtol` as well as `atol` in test_fmha.py (#1743)

  eqy

  authored
  9fc94b7

  Copy full SHA for 9fc94b7

• Loop through all available engines for cuDNN heuristics search (#1740)

  Show description for ddd9b3f
  minitu

  authored
  ddd9b3f

  Copy full SHA for ddd9b3f

Commits on Oct 17, 2023

• Use `torch.testing.assert_close` for better signals (#1742)

  eqy

  authored
  19cc873

  Copy full SHA for 19cc873

Commits on Oct 13, 2023

• add test for openfold triton mha kernel (#1739)

  Show description for 5afc6a0
  azazhu

  authored
  5afc6a0

  Copy full SHA for 5afc6a0

Commits on Oct 3, 2023

• Add openfold triton code (#1734)

  Show description for 58acf96
  arnowaczynski

  authored
  58acf96

  Copy full SHA for 58acf96

Commits on Sep 30, 2023

• Add hysteresis support for AMP gradient scale update (#1733)

  Show description for 6a77872
  

  4 peopleauthored
  6a77872

  Copy full SHA for 6a77872

Commits on Sep 29, 2023

• Distributed optimizer infrastructure for FP8 parameters (#1723)

  Show description for 2386a91
  

  timmoon10

  and

  crcrpar

  authored
  2386a91

  Copy full SHA for 2386a91

Commits on Sep 28, 2023

• Massively reduce LayerNorm/RMSNorm GPU memory usage in modern networks by tricking torch autograd (#1715)

  Show description for 6ff4548
  RuiWang1998

  authored
  6ff4548

  Copy full SHA for 6ff4548

Commits on Sep 19, 2023

• Add multi_tensor_unscale_l2norm_cuda (#1727)

  Show description for 741bdf5
  minitu

  authored
  741bdf5

  Copy full SHA for 741bdf5

Commits on Sep 6, 2023

• Distributed optimizer support for multiple dtypes (#1721)

  Show description for 52e18c8
  timmoon10

  authored
  52e18c8

  Copy full SHA for 52e18c8

Commits on Sep 4, 2023

• [contrib.xentropy] bfloat16 support (#1720)

  Show description for 730376f
  

  crcrpar

  and

  IvanYashchuk

  authored
  730376f

  Copy full SHA for 730376f

Commits on Aug 29, 2023

• Include format version in distopt checkpoints (#1716)

  Show description for 5b04afc
  

  timmoon10

  and

  crcrpar

  authored
  5b04afc

  Copy full SHA for 5b04afc

• Merge pull request #1718 from borisfom/fix-fused-onnx-test

  Show description for 868f996
  borisfom

  authored
  868f996

  Copy full SHA for 868f996

• Update tests/L0/run_fused_layer_norm/test_fused_layer_norm.py

  Show description for 0d47550
  

  borisfom

  and

  crcrpar

  authored
  0d47550

  Copy full SHA for 0d47550

Commits on Aug 28, 2023

• Adjusting test for opset 18

  Show description for a8d9e43
  borisfom

  committed
  a8d9e43

  Copy full SHA for a8d9e43

• Add the warning of distributed_fused_adam low bucket usage (#1714)

  Show description for 1d01e5c
  

  shjwudp

  and

  timmoon10

  authored
  1d01e5c

  Copy full SHA for 1d01e5c

Commits on Aug 22, 2023

• Update GroupNorm for 16 Groups (#1713)

  Show description for bc4be41
  

  alpha0422

  and

  Zhuoyao1012

  authored
  bc4be41

  Copy full SHA for bc4be41

Commits on Aug 19, 2023

• DP-independent checkpoint format for distributed Adam optimizer (#1704)

  Show description for 7995de1
  timmoon10

  authored
  7995de1

  Copy full SHA for 7995de1

Commits on Aug 16, 2023

• skip UccP2PCommTest on single GPU (#1711)

  Aidyn-A

  authored
  38a1269

  Copy full SHA for 38a1269

Commits on Aug 8, 2023

• Add BF16 support to FusedMixedPrecisionLamb (#1439)

  Show description for 6309120
  

  nv-joseli

  and

  crcrpar

  authored
  6309120

  Copy full SHA for 6309120

Commits on Jul 31, 2023

• Add type hints to distributed Adam optimizer (#1699)

  Show description for 2958e06
  timmoon10

  authored
  2958e06

  Copy full SHA for 2958e06