Local SLURM cluster setup

For remote execution of operations on a cluster environment, signac uses the signac-flow interface which allows the users to keep track of submitted operations. If you don’t have access to High-Performance Computing (HPC) clusters then you can set up your local computer to work like a HPC cluster using the open-source SLURM Workload Manager software. To set up a local cluster enviroment with Ubuntu 18.04 and SLURM 19.05.5, follow the steps below.

1. Perform sudo apt update.
2. Install slurm-wlm on your machine using the command sudo apt install slurm-wlm. This installs both the central management daemon slurmctld and the compute node daemon slurmd. For the purposes of this tutorial, we assume that only one computer is part of the cluster and it will be running both the management and the compute node services.
3. You can check the directories where the services are installed using which slurmd and which slurmctld.
4. You can also check the installed version using the command slurmd --version.
5. Execute dpkg -L slurmctld and after that, locate and copy the path of the file named slurm-wlm-configurator.html. In my case it was /usr/share/doc/slurmctld/slurm-wlm-configurator.html. Move to that directory by executing cd /usr/share/doc/slurmctld/
6. Execute python3 -m http.server and open the local web server’s address in your browser.
7. The file slurm-wlm-configurator.html needs to have read permissions, which can be added with chmod +r slurm-wlm-configurator.html. After providing the required permissions to that file, open it.
8. Fill in the text fields according to the requirements and click the submit button. Please note in the ProcessTracking section, there is an option Cgroup (which stands for control groups). Control groups are a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes. However, the control groups feature was not set up on the system I used. Instead, I had to select LinuxProc.
9. After submitting the form, a text file is generated. Save that file to the path /etc/slurm-llnl/slurm.config.
10. You have successfully set up a local Slurm environment! You can start using the service by executing the commands sudo service slurmd start and sudo service slurmctld start. You can also use stop, restart, or status in place of start.