Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Jun 23, 2022

• [transformer] Port Sequence Parallelism (takeover of #1396) (#1400)

  Show description for 3ff1a10
  crcrpar

  authored
  3ff1a10

  Copy full SHA for 3ff1a10

• Move distributed Adam unit test to contrib dir (#1406)

  Show description for 57f890a
  timmoon10

  authored
  57f890a

  Copy full SHA for 57f890a

Commits on Jun 22, 2022

• Temporary Solution to Let `FusedAdam` support BFloat16 (#1407)

  Show description for 81f8ba7
  crcrpar

  authored
  81f8ba7

  Copy full SHA for 81f8ba7

• Gradient clipping with fused kernels (#1405)

  Show description for dcb02fc
  timmoon10

  authored
  dcb02fc

  Copy full SHA for dcb02fc

Commits on Jun 16, 2022

• Remove legacy fuser usage from multihead attention in contrib in favor of the default which should be nvfuser. Modify test scripts to activate fusion. (#1403)

  kevinstephano

  authored
  1403c21

  Copy full SHA for 1403c21

Commits on Jun 14, 2022

• Merge pull request #1401 from timmoon10/dist-adam-zero

  Show description for 5ffb22d
  thorjohnsen

  authored
  5ffb22d

  Copy full SHA for 5ffb22d

• Update documentation to reflect DistributedFusedAdam uses AdamW

  Show description for 846f7f8
  timmoon10

  committed
  846f7f8

  Copy full SHA for 846f7f8

• Update dist Adam test to use updated API

  timmoon10

  committed
  e2af089

  Copy full SHA for e2af089

Commits on Jun 13, 2022

• Add ZeRO-2 support in DistributedFusedAdam

  timmoon10

  committed
  6e41291

  Copy full SHA for 6e41291

Commits on May 31, 2022

• Do pipeline parallelism tests in double because TF32 environment variables can be painful to manage across test suites (#1391)

  Show description for 265b451
  eqy

  authored
  265b451

  Copy full SHA for 265b451

Commits on May 20, 2022

• Add grad check in test pipeline parallel fwd bwd (#1386)

  Show description for ab5fc48
  Aidyn-A

  authored
  ab5fc48

  Copy full SHA for ab5fc48

Commits on May 19, 2022

• Test `len(model) > 1` in `test_pipelining_with_interleaving` (#1384)

  Show description for da1f7f2
  

  eqy

  and

  crcrpar

  authored
  da1f7f2

  Copy full SHA for da1f7f2

• [Pipeline-Parallelism][TF32] Disable TF32 for Pipeline-Parallel numerical checks (#1382)

  Show description for 891d57d
  

  eqy

  and

  crcrpar

  authored
  891d57d

  Copy full SHA for 891d57d

Commits on May 18, 2022

• [transformer] Allow for different backend for Pipeline Parallel ProcessGroups (#1380)

  Show description for 3490b9e
  

  crcrpar

  and

  Aidyn-A

  authored
  3490b9e

  Copy full SHA for 3490b9e

Commits on May 13, 2022

• Change the timing of deprecation warning (#1383)

  crcrpar

  authored
  d36397d

  Copy full SHA for d36397d

Commits on May 12, 2022

• Async pipeline parallel (#1373)

  Show description for 3fe3521
  eqy

  authored
  3fe3521

  Copy full SHA for 3fe3521

Commits on May 11, 2022

• [transformer] add loss comparison to test_pipeline_parallel_fwd_bwd (#1374)

  Show description for 6844026
  Aidyn-A

  authored
  6844026

  Copy full SHA for 6844026

Commits on Apr 29, 2022

• [transformer][pipeline parallel] fix typo in test (#1370)

  Show description for c3018b1
  eqy

  authored
  c3018b1

  Copy full SHA for c3018b1

• [transformer][pipeline parallel] warn if deallocation is enabled (#1365)

  Show description for 2b7d280
  crcrpar

  authored
  2b7d280

  Copy full SHA for 2b7d280

• [FastLayerNorm] Support hidden dim of 14336 (#1368)

  yjk21

  authored
  77f9d73

  Copy full SHA for 77f9d73

Commits on Apr 21, 2022

• Give Some Extensions Version Guard in Build&Runtime (#1358)

  Show description for f9305e7
  crcrpar

  authored
  f9305e7

  Copy full SHA for f9305e7

Commits on Apr 20, 2022

• Merge pull request #1340 from NVIDIA/peer_memory

  Show description for fed20d2
  thorjohnsen

  authored
  fed20d2

  Copy full SHA for fed20d2

Commits on Apr 19, 2022

• [submodule update] Bump cudnn-frontend to v0.6.1 (#1353)

  Show description for d89f5e6
  crcrpar

  authored
  d89f5e6

  Copy full SHA for d89f5e6

Commits on Apr 14, 2022

• Bit faster

  thorjohnsen

  committed
  5698eee

  Copy full SHA for 5698eee

Commits on Apr 13, 2022

• Bug fixes

  thorjohnsen

  committed
  140282d

  Copy full SHA for 140282d

Commits on Apr 8, 2022

• Add graphing, switch to peer mem exchanger as default

  thorjohnsen

  committed
  bec558b

  Copy full SHA for bec558b

• Bug fix

  thorjohnsen

  committed
  4aeb24c

  Copy full SHA for 4aeb24c

• Fix deadlock issue when peer memory halo exchanger is used with cuda graph

  thorjohnsen

  committed
  c70f0e3

  Copy full SHA for c70f0e3

Commits on Apr 7, 2022

• Deprecation warning: `pyprof` & `reparameterization` (#1348)

  Show description for 727a645
  crcrpar

  authored
  727a645

  Copy full SHA for 727a645

• [transformer] add microbatches test (#1349)

  Show description for 7d90387
  crcrpar

  authored
  7d90387

  Copy full SHA for 7d90387

Commits on Apr 5, 2022

• Rename nccl_p2p extension to nccl_p2p_cuda

  thorjohnsen

  committed
  d8db8c1

  Copy full SHA for d8db8c1

• Rename peer_memory extension to peer_memory_cuda

  thorjohnsen

  committed
  6e7e2d9

  Copy full SHA for 6e7e2d9

Commits on Apr 3, 2022

• Clean up code

  thorjohnsen

  committed
  fa8e7d9

  Copy full SHA for fa8e7d9

Commits on Apr 2, 2022

• Bug fix

  thorjohnsen

  committed
  05dd9c6

  Copy full SHA for 05dd9c6

• Bug fix

  thorjohnsen

  committed
  a5d51c0

  Copy full SHA for a5d51c0