Commits

Branch selector

 master

User selector

All users

All time

Commit History

Commits on Nov 10, 2021

• conditionally import amp_C (#1211)

  crcrpar

  authored
  25bfcb9

  Copy full SHA for 25bfcb9

• check in (#1210)

  eqy

  authored
  2205cff

  Copy full SHA for 2205cff

• check in (#1205)

  eqy

  authored
  fa8bd7e

  Copy full SHA for fa8bd7e

Commits on Oct 27, 2021

• `FastLayerNorm` compat with `autocast` (#1203)

  Show description for ae75763
  

  crcrpar

  and

  yjk21

  authored
  ae75763

  Copy full SHA for ae75763

• Pipeline Model Parallel (#1202)

  Show description for 63d5dd6
  

  3 peopleauthored
  63d5dd6

  Copy full SHA for 63d5dd6

Commits on Oct 23, 2021

• Use out-of-place to avoid D2D copy in tensor parallel cross entropy (#1198)

  Show description for 3303b3e
  

  crcrpar

  and

  eqy

  authored
  3303b3e

  Copy full SHA for 3303b3e

Commits on Oct 18, 2021

• remove THC headers/functions (#1192)

  Show description for 0c7d8e3
  crcrpar

  authored
  0c7d8e3

  Copy full SHA for 0c7d8e3

Commits on Oct 16, 2021

• replace (#1191)

  crcrpar

  authored
  60821f5

  Copy full SHA for 60821f5

Commits on Oct 14, 2021

• change chunking scheme for full-allreduce case, add parameter order argument, both to enable contiguous chunking of allgather (#1190)

  seryilmaz

  authored
  1d5f7e5

  Copy full SHA for 1d5f7e5

• Fix dist lamb (#1185)

  Show description for d9a46fd
  nanz-nv

  authored
  d9a46fd

  Copy full SHA for d9a46fd

Commits on Oct 13, 2021

• check in (#1189)

  eqy

  authored
  4e9fae9

  Copy full SHA for 4e9fae9

Commits on Oct 8, 2021

• Remove `custom_fwd`/`custom_bwd` from fused softmax (#1188)

  Show description for 14ccf59
  crcrpar

  authored
  14ccf59

  Copy full SHA for 14ccf59

• check in (#1187)

  eqy

  authored
  3ad9db2

  Copy full SHA for 3ad9db2

Commits on Oct 7, 2021

• Update layer_norm_cuda_kernel.cu (#1184)

  eqy

  authored
  5adf7bc

  Copy full SHA for 5adf7bc

Commits on Oct 6, 2021

• ColumnParallelLinearWithAsyncAllreduce autocast support (#1183)

  Show description for b3da603
  crcrpar

  authored
  b3da603

  Copy full SHA for b3da603

Commits on Oct 2, 2021

• transformer utils (#1181)

  Show description for 365fdc1
  

  5 peopleauthored
  365fdc1

  Copy full SHA for 365fdc1

Commits on Sep 30, 2021

• use cuda caching allocator from pytorch (#1180)

  xwang233

  authored
  bdac244

  Copy full SHA for bdac244

Commits on Sep 28, 2021

• cleanup missing THCDeviceUtils.cuh header (#1177)

  xwang233

  authored
  2a559c5

  Copy full SHA for 2a559c5

Commits on Sep 24, 2021

• Fix typo in contrib FusedLamb. (#1172)

  romerojosh

  authored
  70d4a0b

  Copy full SHA for 70d4a0b

• THCDeviceUtils.cuh -> ATen/cuda/DeviceUtils.cuh (#1173)

  crcrpar

  authored
  76daa45

  Copy full SHA for 76daa45

Commits on Sep 8, 2021

• enable ninja (#1164)

  Show description for 9ce0a10
  crcrpar

  authored
  9ce0a10

  Copy full SHA for 9ce0a10

Commits on Sep 4, 2021

• fix CUBLAS guards (#1162)

  Show description for 54b9391
  

  seryilmaz

  and

  Sukru Eryilmaz

  authored
  54b9391

  Copy full SHA for 54b9391

Commits on Sep 2, 2021

• Merge pull request #1161 from NVIDIA/optional_caller_supplied_communicator

  Show description for ae1cdd6
  thorjohnsen

  authored
  ae1cdd6

  Copy full SHA for ae1cdd6

• Optional NCCL communicator argument to init method

  thorjohnsen

  committed
  e777bdd

  Copy full SHA for e777bdd

• Merge pull request #1160 from NVIDIA/bug_fix_in_wgrad

  Show description for 9b88066
  thorjohnsen

  authored
  9b88066

  Copy full SHA for 9b88066

• Bug fix in wgrad

  thorjohnsen

  committed
  9e29572

  Copy full SHA for 9e29572

• Merge pull request #1159 from NVIDIA/more_bug_fixes

  Show description for 0506fe3
  thorjohnsen

  authored
  0506fe3

  Copy full SHA for 0506fe3

• Revert some changes

  thorjohnsen

  committed
  8c4a007

  Copy full SHA for 8c4a007

• Bug fixes

  thorjohnsen

  committed
  8cdcc82

  Copy full SHA for 8cdcc82

• Merge pull request #1158 from NVIDIA/bug_fixes

  Show description for 0cb1cb3
  thorjohnsen

  authored
  0cb1cb3

  Copy full SHA for 0cb1cb3

• More detailed output

  thorjohnsen

  committed
  67a0ffc

  Copy full SHA for 67a0ffc

• Bug fixes

  thorjohnsen

  committed
  bc9114c

  Copy full SHA for bc9114c

• option to set param views to flat buffer (#1152)

  Show description for 17eec27
  

  3 peopleauthored
  17eec27

  Copy full SHA for 17eec27

• use prescaling for collective (#1157)

  seryilmaz

  authored
  2e98baa

  Copy full SHA for 2e98baa

• Add full all-reduce code path for DistributedFusedAdam (#1146)

  Show description for 1cb9c5c
  

  kexinyu

  and

  ptrblck

  authored
  1cb9c5c

  Copy full SHA for 1cb9c5c