HPC Job Dependency

SLURM allows jobs to depend on the status of prior jobs, enables submitting jobs which starts after its dependent jobs completed correctly.

Job Dependency Type

Dependencies that can be used for <type> and <previous_job_id>

Type	Description
after	This job can begin execution after the specified jobs have begun execution
afterany	This job can begin execution after the specified jobs have terminated.
aftercorr	A task of this job array can begin execution after the corresponding task ID in the specified job has completed successfully
afternotok	This job can begin execution after the specified jobs have terminated in some failed state
afterok	This job can begin execution after the specified jobs have successfully executed
singleton	This job can begin execution after any previously launched jobs sharing the same job name and user have terminated

Example Command

Job that only runs after successful completion of the first job as follows where we set the afterok as the dependency type

$ sbatch --dependency=afterok:<previous_job_id> <job_script>.sh

If the job requires more than one job to be completed before it is executed, you can supply all the job ids using , separator

$ sbatch --dependency=<type>:<previous_job_id>,<previous_job_id>,<previous_job_id> <job_script>.sh

You can also set the job to run if any one of the job ids completes successfully using a ? Separator

$ sbatch --dependency=<type>:<previous_job_id>?<previous_job_id>?<previous_job_id>? <job_script>.sh