Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Oct 26, 2025

• modify tensor.type() to tensor.options()

  Source-Roc

  committed
  9821ea5

  Copy full SHA for 9821ea5

• Finish cleaning up usage of VERSION_GE_1_1/1_3/1_5 macros

  Source-Roc

  committed
  82c4224

  Copy full SHA for 82c4224

Commits on Aug 21, 2025

• Remove version dependent macros & extension specific arch flags except fmha (#1918)

  crcrpar

  authored
  26bba57

  Copy full SHA for 26bba57

Commits on Nov 1, 2024

• `Tensor.type()` -> `Tensor.scalar_type()` (#1855)

  Show description for e552ad9
  crcrpar

  authored
  e552ad9

  Copy full SHA for e552ad9

Commits on Aug 17, 2024

• Fix illegal memory access with multi_tensor_apply size above INT_MAX (#1825)

  Show description for 79e3dc4
  gdb

  authored
  79e3dc4

  Copy full SHA for 79e3dc4

Commits on Jul 2, 2024

• release gil (#1816)

  Show description for 23c1f86
  crcrpar

  authored
  23c1f86

  Copy full SHA for 23c1f86

Commits on Apr 26, 2024

• Fix reduce_blocks_into_lanes race condition (#1798)

  Show description for a7de60e
  Fuzzkatt

  authored
  a7de60e

  Copy full SHA for a7de60e

Commits on Apr 19, 2024

• Add 2D Fused RoPE (#1784)

  Show description for b5df1cc
  yaox12

  authored
  b5df1cc

  Copy full SHA for b5df1cc

Commits on Jan 12, 2024

• Fused RoPE for `thd` format (#1756)

  Show description for f058162
  yaox12

  authored
  f058162

  Copy full SHA for f058162

Commits on Jan 5, 2024

• 64-bit indexing Adam (#1765)

  Show description for 87c4deb
  eqy

  authored
  87c4deb

  Copy full SHA for 87c4deb

Commits on Nov 29, 2023

• [FusedRoPE] Fuse type conversion and cos/sin (#1752)

  Show description for 37d83fc
  yaox12

  authored
  37d83fc

  Copy full SHA for 37d83fc

Commits on Nov 23, 2023

• Avoid `.contiguous()` in fused RoPE (#1751)

  Show description for dc5fa38
  yaox12

  authored
  dc5fa38

  Copy full SHA for dc5fa38

Commits on Nov 16, 2023

• fix a bug in fused rope (#1750)

  Show description for fd4ae7d
  yaox12

  authored
  fd4ae7d

  Copy full SHA for fd4ae7d

Commits on Nov 14, 2023

• A fused `apply_rotary_pos_emb` implementation for Megatron-Core (#1746)

  Show description for 08f7402
  yaox12

  authored
  08f7402

  Copy full SHA for 08f7402

Commits on Sep 30, 2023

• Add hysteresis support for AMP gradient scale update (#1733)

  Show description for 6a77872
  

  4 peopleauthored
  6a77872

  Copy full SHA for 6a77872

Commits on Sep 28, 2023

• Massively reduce LayerNorm/RMSNorm GPU memory usage in modern networks by tricking torch autograd (#1715)

  Show description for 6ff4548
  RuiWang1998

  authored
  6ff4548

  Copy full SHA for 6ff4548

Commits on Sep 19, 2023

• Add multi_tensor_unscale_l2norm_cuda (#1727)

  Show description for 741bdf5
  minitu

  authored
  741bdf5

  Copy full SHA for 741bdf5

Commits on Aug 8, 2023

• Add BF16 support to FusedMixedPrecisionLamb (#1439)

  Show description for 6309120
  

  nv-joseli

  and

  crcrpar

  authored
  6309120

  Copy full SHA for 6309120

Commits on Jun 1, 2023

• tensor.type() is deprecated, we should use scalar_type() (#1672)

  Show description for ecb5614
  

  ycsos

  and

  yuchuang

  authored
  ecb5614

  Copy full SHA for ecb5614

Commits on Apr 18, 2023

• add bf16 support for FusedDense (#1627)

  Agoniii

  authored
  1ba89ee

  Copy full SHA for 1ba89ee

Commits on Apr 14, 2023

• Add support for FP32 master weights in FusedAdam optimizer (#1625)

  Show description for 6bd01c4
  

  4 peopleauthored
  6bd01c4

  Copy full SHA for 6bd01c4

Commits on Mar 22, 2023

• Revert "Add support for FP32 master weights in FusedAdam optimizer (#1623)" (#1624)

  Show description for 4060aef
  crcrpar

  authored
  4060aef

  Copy full SHA for 4060aef

• Add support for FP32 master weights in FusedAdam optimizer (#1623)

  Show description for 6952004
  

  3 peopleauthored
  6952004

  Copy full SHA for 6952004

Commits on Mar 7, 2023

• s/maximimize/maximize/ (#1604)

  Show description for be819ea
  EIFY

  authored
  be819ea

  Copy full SHA for be819ea

Commits on Feb 25, 2023

• Decouple optimizer state and grad dtypes in distributed Adam optimizer (#1575)

  Show description for 03c9d80
  timmoon10

  authored
  03c9d80

  Copy full SHA for 03c9d80

Commits on Feb 9, 2023

• decrease warnings & remove append_nvcc_threads from setup.py (#1581)

  Show description for 0145d69
  crcrpar

  authored
  0145d69

  Copy full SHA for 0145d69

Commits on Feb 3, 2023

• [fused softmax] integer overflow and non-contiguous memory (#1568)

  Show description for 5c9625c
  cyanguwa

  authored
  5c9625c

  Copy full SHA for 5c9625c

Commits on Jan 31, 2023

• Capturable Adam optimizer (#1560)

  Show description for ad68a2f
  gilshm

  authored
  ad68a2f

  Copy full SHA for ad68a2f

Commits on Jan 19, 2023

• Extend softmax fusion seq length to 16384 (#1558)

  Show description for e1e0531
  yaoyu-33

  authored
  e1e0531

  Copy full SHA for e1e0531

Commits on Nov 22, 2022

• Update megatron fused softmax follow megatron-lm (#1539)

  Show description for abeca58
  yaoyu-33

  authored
  abeca58

  Copy full SHA for abeca58

Commits on Aug 25, 2022

• Optimize distributed Adam kernels and implementation (#1463)

  Show description for 6f7d8ac
  timmoon10

  authored
  6f7d8ac

  Copy full SHA for 6f7d8ac

Commits on Aug 5, 2022

• Add a generic fused softmax (#1440)

  Show description for cd0a1f1
  yidong72

  authored
  cd0a1f1

  Copy full SHA for cd0a1f1

Commits on Aug 4, 2022

• support for full mask in fused softmax kernel + avoiding inplace operation in bwd pass (#1444)

  Show description for 31cbdd1
  kvareddy

  authored
  31cbdd1

  Copy full SHA for 31cbdd1

Commits on Jun 22, 2022

• Temporary Solution to Let `FusedAdam` support BFloat16 (#1407)

  Show description for 81f8ba7
  crcrpar

  authored
  81f8ba7

  Copy full SHA for 81f8ba7

Commits on Feb 26, 2022

• [transformer] Fuse grad accumulation with wgrad (#1297)

  Show description for ddc0803
  

  crcrpar

  and

  erhoo82

  authored
  ddc0803

  Copy full SHA for ddc0803