Accelerate ND-Parallel: A guide to Efficient Multi-GPU Training

Why it matters

The timing matters: accelerate is converging with shifts in ndparallel, which could amplify the downstream impact.