About the choice of batch size and learning rate #13
Description
When I train on 8 * A100-80G GPUs, my batchsize can be set to 51200. Does the learning rate need to be adjusted accordingly? Will setting such a large batchsize affect training?
{{ message }}