Slurm Examples

As the saying goes, talk is cheap, show me the code! This section provides practical examples of using Slurm for various workloads, with a focus on distributed training using PyTorch. These examples are designed to be easily adaptable to your specific use cases and cluster environments.

Important

The examples provided here are based on Amii’s Vulcan cluster and may require adjustments to work on other clusters. Always refer to your cluster’s documentation and consult with your system administrators for any cluster-specific configurations or limitations.

• Introduction to PyTorch DDP
  • Advantages of DDP
  • When to use DDP
  • Show me the code!
  • First run on Slurm
  • Summary
• Introduction to PyTorch FSDP
  • Advantages of FSDP
  • When to use FSDP
  • Show me the code!
  • First run on Slurm
  • Summary