Skip to content

HPC submission scripts

We recommend running your batch jobs using submission scripts, as this makes it easier to see and edit your commands, and to re-run your job with modified parameters.

Script template

Here is template you can follow:

#!/bin/bash  

# Include your job submission details as #BSUB headers
#BSUB -q <your_queue>  
#BSUB -P <yourProject>  
#BSUB -o <path_to/job.%J.out>  
#BSUB -e <path_to/job.%J.err>  
#BSUB -J <jobName>  
#BSUB -R "rusage[mem=1000] span[hosts=1]"  
#BSUB -M <max_memory_in_MB>
#BSUB -n <number_of_cores>  
#BSUB -cwd <"your_dir">  

# Set your temp directory as the re_scratch folder
export TMPDIR=/re_scratch/re_gecip/<your_GECIP>/<your_username>  
export TMPDIR=/re_scratch/re_discovery_forum/<your_discovery_forum_folder>/<your_username>

# Load any required modules from the HPC
module load <moduleName>  

# The actual script you want to run

Job submission options

option definition example input
-q The queue you want to run your job in -q short
-q medium
-q long
-P Your LSF project code -P re_gecip_cancer_breast
-P re_df_illumina
-o Where you want your standard output to go, in your working folder /re_gecip/re_gecip_cancer_breast/jane_doe_analysis/job.%J.out
discovery_forum/re_df_illumina/john_doe_work/job.%J.out
-e Where you want your standard error to go, in your working folder /re_gecip/re_gecip_cancer_breast/jane_doe_analysis/job.%J.err
discovery_forum/re_df_illumina/john_doe_work/job.%J.err
-J A name for your job, with no spaces script_test_1
analysis-breast-cancer
-R "rusage[mem=###] span[hosts=#]" The memory you wish to allocate to your job, where mem is the memory allocation in MB and hosts is the number of cores to use. -R rusage[mem=500]
-M The maximum memory for your job to use -M 500
-n The number of cores for your job to use. -n 2
-cwd Your working directory /re_gecip/re_gecip_cancer_breast/jane_doe_analysis
discovery_forum/re_df_illumina/john_doe_work

Temporary files

We recommend that you use the re_scratch folder for your temporary files. These folders are not included in our regular back-ups, so this prevents using up finite space.

Load any required modules from the HPC

You can make use of any of the modules installed on the HPC. Load these here, making sure you have previously run module avail in the terminal to get the correct path.

This can include nextflow for pipelines, and singularity for using containers.

Submitting your script

Use the following command to submit the script:

bsub < myscript.sh

The < sign indicates that your script is fed into the submission command.