Top tips when working with SLURM
- Put all your SLURM directives at the top of the script file, above any commands. Any directive after an executable line in the script is ignored.
- Note that you can pass SLURM directives as options to the
sbatchcommand. - Check man sbatch for a complete reference of all the options and their description.
Main command line tools
The table summarises the main Slurm commands and their PBSpro equivalents.
User commands | PBSpro | Slurm |
|---|---|---|
| Job submission | qsub [<pbs_options>] <job_script> | sbatch [<sbatch_options>] <job_script> |
| Job cancellation | qdel <job_id> | scancel <job_id> |
| Job status | qscan [-u <uid>] [<job_id>] | squeue [-u <uid>] -j [<job_id>] |
| Queue information | qstat -Q [-f] [<queue>] | squeue [<squeue_options>] |
Main directives
This table shows the main directives used in both Slurm and PBSpro side by side
| Keyword | PBSpro | Slurm | Notes |
|---|---|---|---|
| Prefix | #PBS | #SBATCH | |
| Queue | -q <queue> |
| |
| Job Name |
|
| |
| Shell | -S /usr/bin/ksh | #!/usr/bin/ksh | Slurm uses the shebang of the script as the shell to interpret it. |
| Wall-clock limit | -l walltime=<hh:mm:ss> |
| |
| Initial working directory | no equivalent |
| |
| Job output | -o <output_file> |
| |
| Job error | -e <error_file> |
| |
| Email notification | -m <event> | --mail-type=<type> | Type can be NONE, BEGIN, END, FAIL, REQUEUE, ALL |
| Email user | -M <email> | --mail-user=<email> | |
| Set environment variables | -v <ENV1>=<value1>, <ENV2>=<value2> | --export=<ENV1>=<value1>,<ENV2>=<value2> | |
| Copy environment | -V | --export=ALL | The default on this platform is to copy no environment i.e. start a fresh session. |
| Number of nodes | -l EC_nodes=<nodes> |
| |
| Total number of MPI tasks | -l EC_total_tasks=<tasks> |
| |
| Number of parallel threads per MPI task | -l EC_threads_per_task=<threads> |
| |
| Number of MPI tasks per node | -l EC_tasks_per_node=<tasks> |
| |
| Consumable Memory per MPI task | -l EC_memory_per_task=<memory> | --mem-per-cpu=<memory> | |
| Use hyperthreading / SMT | -l EC_hyperthreads=2 / 1 | --threads-per-core=<threads> | |
| Billing account | -l EC_billing_account=<account> |
| |
| TMPDIR size | -l EC_job_tmpdir=<size>G |
| Only relevant to shared nodes (*f) QoSs. TMPDIR is memory based on Cray, but SSD-backed on Atos |