Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on May 26, 2021

• Distributed LAMB: Clip grads before reduce_scatter/all_reduce (#1099)

  Show description for ebcd7f0
  kexinyu

  authored
  ebcd7f0

  Copy full SHA for ebcd7f0

Commits on May 17, 2021

• compile cublasLt code only for cublas >= 11.0 (#1108)

  Show description for 00c1e56
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  00c1e56

  Copy full SHA for 00c1e56

Commits on Apr 19, 2021

• Fix cublasLt context create/destroy overhead in MLP extension (#1083)

  Show description for 082f999
  seryilmaz

  authored
  082f999

  Copy full SHA for 082f999

Commits on Apr 17, 2021

• initial cublaslt support for MLP (#1080)

  Show description for b8be1bc
  

  seryilmaz

  and

  ptrblck

  authored
  b8be1bc

  Copy full SHA for b8be1bc

• remove MIT license (#1081)

  ptrblck

  authored
  b5eb38d

  Copy full SHA for b5eb38d

• Adding fast bottleneck implementation into contrib (#1079)

  Show description for 705cba9
  

  FDecaYed

  and

  ptrblck

  authored
  705cba9

  Copy full SHA for 705cba9

Commits on Apr 16, 2021

• adds fmhalib (#1074)

  yjk21

  authored
  5c9b21d

  Copy full SHA for 5c9b21d

Commits on Apr 15, 2021

• Update README.md (#1067)

  Show description for e5f2f67
  jayrodge

  authored
  e5f2f67

  Copy full SHA for e5f2f67

• DistributedFusedLAMB: enable no_copy and add barrier for SHARP (#1075)

  Show description for bb79158
  

  kexinyu

  and

  Kexin Yu

  authored
  bb79158

  Copy full SHA for bb79158

• Add unit tests for Fused NovoGrad (#1065)

  Show description for 59d2f7a
  sudhakarsingh27

  authored
  59d2f7a

  Copy full SHA for 59d2f7a

Commits on Mar 24, 2021

• sync-free Distributed LAMB + parameter reordering (#1055)

  Show description for a651e2c
  

  kexinyu

  and

  Kexin Yu

  authored
  a651e2c

  Copy full SHA for a651e2c

• Initial check-in of the transducer extensions (#1069)

  Show description for d86d1b0
  nanz-nv

  authored
  d86d1b0

  Copy full SHA for d86d1b0

Commits on Feb 23, 2021

• fast layer norm (#1037)

  yjk21

  authored
  e2083df

  Copy full SHA for e2083df

Commits on Feb 10, 2021

• fix import container_abcs issue (#1049)

  Show description for a78ccf0
  ShoufaChen

  authored
  a78ccf0

  Copy full SHA for a78ccf0

Commits on Jan 20, 2021

• cuda rng changes for graph capture with apex MHA (#1025)

  Show description for eefb1ba
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  eefb1ba

  Copy full SHA for eefb1ba

Commits on Dec 17, 2020

• Merge pull request #1015 from jpool-nv/patch-1

  Show description for 154c633
  thorjohnsen

  authored
  154c633

  Copy full SHA for 154c633

• Update ASP README to highlight default recipe

  Show description for 56914d4
  jpool-nv

  authored
  56914d4

  Copy full SHA for 56914d4

Commits on Dec 4, 2020

• remove noise pip-version-check noise that hides the outcome of the build (#998)

  stas00

  authored
  8cf5ae6

  Copy full SHA for 8cf5ae6

• Distributed LAMB fixes (#1007)

  Show description for 8a80d47
  

  3 peopleauthored
  8a80d47

  Copy full SHA for 8a80d47

• Seryilmaz/fused dropout softmax (#985)

  Show description for 3fe10b5
  seryilmaz

  authored
  3fe10b5

  Copy full SHA for 3fe10b5

Commits on Dec 2, 2020

• Fix lack of proper loading of best_prec1 from the checkpoint (#1000)

  Show description for 6c186b3
  JanuszL

  authored
  6c186b3

  Copy full SHA for 6c186b3

Commits on Dec 1, 2020

• DistributedFusedAdam Model Parallelism Support (Megatron) (#981)

  Show description for 6b7e77b
  

  3 peopleauthored
  6b7e77b

  Copy full SHA for 6b7e77b

Commits on Oct 19, 2020

• Optimize the sync batchnorm by batching the communication (#980)

  Show description for 8a1ed9e
  lly-zero-one

  authored
  8a1ed9e

  Copy full SHA for 8a1ed9e

Commits on Sep 29, 2020

• use reshape instead of view (#971)

  ptrblck

  authored
  a109f85

  Copy full SHA for a109f85

Commits on Sep 15, 2020

• Merge pull request #959 from a-maci/update-ASP-readme

  Show description for 4a1fa2c
  thorjohnsen

  authored
  4a1fa2c

  Copy full SHA for 4a1fa2c

Commits on Sep 14, 2020

• Merge pull request #5 from a-maci/a-maci-patch-update-asp-readme

  Show description for 48fc613
  a-maci

  authored
  48fc613

  Copy full SHA for 48fc613

• Update README for ASP

  Show description for e3794f4
  a-maci

  authored
  e3794f4

  Copy full SHA for e3794f4

Commits on Aug 15, 2020

• Should pass stricter stride/size checks in pytorch (#942)

  

  mcarilli

  authored
  4ef930c

  Copy full SHA for 4ef930c

Commits on Aug 10, 2020

• move sm80 code inside MHA (#937)

  Show description for 5d9b5cb
  ptrblck

  authored
  5d9b5cb

  Copy full SHA for 5d9b5cb

Commits on Aug 6, 2020

• sgd supports zero-grad (#926)

  

  ngimel

  authored
  85b1783

  Copy full SHA for 85b1783

Commits on Aug 5, 2020

• set device guard for multi tensor optimizer implementations (#927)

  Show description for 274cc06
  

  ngimel

  authored
  274cc06

  Copy full SHA for 274cc06

Commits on Aug 1, 2020

• Add sm80 for CUDA >= 11 (#925)

  ptrblck

  authored
  5b53121

  Copy full SHA for 5b53121

Commits on Jul 30, 2020

• fix default mode missing additive mask option (#924)

  Show description for 700d682
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  700d682

  Copy full SHA for 700d682

Commits on Jul 23, 2020

• Merge pull request #918 from a-maci/ASP_sparse_param_dict_update

  Show description for 459de22
  thorjohnsen

  authored
  459de22

  Copy full SHA for 459de22

Commits on Jul 22, 2020

• Merge pull request #4 from a-maci/ASP_sparse_param_dict_update

  Show description for eb5e96c
  a-maci

  authored
  eb5e96c

  Copy full SHA for eb5e96c