Set Up and Test MSCCL

This article contains some straightforward instructions for getting MSCCL up and running on a cluster. It assumes that lmod contains the necessary dependencies (CUDA, MPI, etc.). If you’re running on a cluster without lmod, you may need to install these dependencies manually.

Build Parts

The first part of this article will be focused on building all parts necessary for MSCCL.

Set Environment Variables

If you’re running on a cluster with Slurm, you can load the CUDA module:

Git Basics

Git was originally created by Linus Torvalds (also the creator of Linux) in 2005 to replace the BitKeeper software and help with development of the Linux kernel. It was obviously designed with this in mind, and works best with source code and text files. Git is maintained independent from the Linux kernel, and is not managed by private companies like GitHub or Gitlab.

Benefits of Using a Version Control System (VCS)

With a VCS, we can easily do the following things: