Commits

Breadcrumbs

History for

onmaster

User selector

All users

All time

Commit History

Commits on Mar 4, 2026

• Deprecate apex.contrib.fmha and apex.contrib.multihead_attn (#1932)

  Show description for 212061e
  

  3 peopleauthored
  212061e

  Copy full SHA for 212061e

Commits on Nov 28, 2025

• Enable Ruff pre-commit hooks (#1957)

  Show description for 16fadfe
  

  3 peopleauthored
  16fadfe

  Copy full SHA for 16fadfe

Commits on Feb 13, 2023

• replace torch.Tensor with torch.empty (#1578)

  Show description for ba027dd
  NouamaneTazi

  authored
  ba027dd

  Copy full SHA for ba027dd

Commits on Jul 14, 2022

• [contrib] Fix the reference implementation of multihead_attn (#1423)

  Show description for 809043f
  crcrpar

  authored
  809043f

  Copy full SHA for 809043f

Commits on Jun 16, 2022

• Remove legacy fuser usage from multihead attention in contrib in favor of the default which should be nvfuser. Modify test scripts to activate fusion. (#1403)

  kevinstephano

  authored
  1403c21

  Copy full SHA for 1403c21

Commits on Dec 14, 2021

• Faster `--fast_multihead_attn` build (#1245)

  Show description for 7ec8ed6
  crcrpar

  authored
  7ec8ed6

  Copy full SHA for 7ec8ed6

Commits on Dec 9, 2021

• Remove `THCState` from `apex/contrib/multihead_attn` (#1239)

  Show description for 1203099
  crcrpar

  authored
  1203099

  Copy full SHA for 1203099

Commits on Dec 4, 2020

• Seryilmaz/fused dropout softmax (#985)

  Show description for 3fe10b5
  seryilmaz

  authored
  3fe10b5

  Copy full SHA for 3fe10b5

Commits on Jul 30, 2020

• fix default mode missing additive mask option (#924)

  Show description for 700d682
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  700d682

  Copy full SHA for 700d682

Commits on Jul 9, 2020

• Fixed weight init for fused weight matrices in fused MHA by adding correct gain factor.

  szmigacz

  committed
  a0d99fd

  Copy full SHA for a0d99fd

Commits on May 29, 2020

• Fixes to Multihead Attention with LayerNorm and Dropout-Add (#860)

  kevinstephano

  authored
  5754fa7

  Copy full SHA for 5754fa7

• Fuses dropout and softmax in backward pass, add bias support to CPP MHA, add additive mask support, separate Q/K/V parameters (#854)

  Show description for 6c2babf
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  6c2babf

  Copy full SHA for 6c2babf

Commits on Apr 30, 2020

• fix dropout scaling from p to 1/(1-p) (#816)

  Show description for aad9300
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  aad9300

  Copy full SHA for aad9300

Commits on Feb 24, 2020

• Change to Multihead Attention to allow Batched GEMMs larger than 64K. (#728)

  Show description for 1733946
  kevinstephano

  authored
  1733946

  Copy full SHA for 1733946

Commits on Feb 6, 2020

• Add Fast Multihead Attention to APEX Contrib (#697)

  Show description for 3f94528
  kevinstephano

  authored
  3f94528

  Copy full SHA for 3f94528

Loading