What is an HPC?¶
A High Performance Cluster (HPC) is a way to carry out large-scale analysis, using centralised compute.
flowchart TD
A(Researcher submits job) --> B[Master host and candidates]
B --> C[Queues]
C --> |jobs wait in queues until the required resources are ready| D[Resources]
D <-.-> |Master host and resources are in frequent communication| B
D --> E(Job runs and finishes)
classDef researcher fill:#DF007D,stroke:#DF007D,color:#FFFFFF;
class A,E researcher;
classDef RE fill:#FFC6E6,stroke:#FFC6E6,color:#2B2F3B;
class B,C,D RE;
Overview of usage¶
To use the HPC, you start by logging onto the cluster. This brings you to the login node. From here you can cd
into your working folder. It is possible to load and run software from the login node, but to make use of the full compute, you should launch jobs.
%%{init: {"flowchart": {"htmlLabels": false, 'curve': 'linear'}} }%%
flowchart TB
subgraph "`RE`"
direction TB
B["Research Environment"] --> C["Terminal"]
end
subgraph "`HPC`"
direction TB
D["`**Login node** low resourced`"] -- "`create job`" --> E["`**Worker node** high resourced`"]
end
subgraph "`Weka`"
direction LR
F[Weka storage] --> G["`**discovery_forum:** read/write folder for Discovery Forum members`"]
F --> H["`**re_gecip:** read/write folder for Research Network members`"]
F --> I["`**genomes:** read only, contains all consented genomes`"]
F --> J["`**public_data_resources:** read only, contains public resources, eg gnomAD`"]
F --> K["`**gel_data_resources:** read only, contains GEL-generated datasets, eg AggV2`"]
end
A("`Researcher`") --> RE
C -- "`ssh`" --> HPC
HPC --> Weka
RE --> Weka
classDef node fill:#FFC6E6,stroke:#FFC6E6,color:#2B2F3B;
class A,B,C,D,E,F,G,H,I,J,K node;
Terminology¶
Term | Meaning |
---|---|
LSF | Load Sharing Facility - the tool we use to schedule jobs on the HPC |
CPU | Central Processing Unit, the main processors |
Nodes | Ephemeral storage, networking, memory and processing resources that can be consumed by virtual machine instances. Sometimes referred to as hosts . |
Job | A task that you run on the HPC. Jobs can spawn other jobs. |
Queue | When you submit a job, it joins a queue. You can choose which queue to join, depending on the length of the job. |
Batch jobs | A job that you set off, then it runs independently in the background |
Interactive jobs | A job that opens access to the HPC, allowing you to run commands and tools on the cluster |
Running | A job that is in progress |
Pending | A job that is waiting in the queue |
working directory | The folder where you put all your files. |
standard output | Information about the job as it runs. If you were running a job normally, this would appear in the terminal, however on an HPC, you should set a file to write this to. |
standard error | Information about errors from the job. If you were running a job normally, this would appear in the terminal, however on an HPC, you should set a file to write this to. |
scratch | A location to write any temporary files creating during the job. |
project code | researchers are grouped based on Discovery Forum and Research Network membership, with compute resources shared between the groups |
modules | Software that has been loaded onto the HPC, which you can use in your analysis |
Usage guidelines¶
DO
DO launch interactive jobs to run software on the HPC.
DO kill interactive jobs when you've finished with them.
DO choose the appropriate length queue for your job.
DO estimate the memory required for your job.
DO use scripts to launch your batch jobs.
DO set LSF parameters (#BSUB
) within your scripts for improved traceability in your batch jobs.
DO specify the location for your standard output and error to help with troubleshooting
DO use scratch directories for your temporary files.
DO use containers to import software.
DO work with interactive coding tools such as Rstudio and Jupyter on the HPC.
DO set up .netrc
to use the LabKey API.
DON'T
DON'T run software on the login node.
DON'T request more memory than you need.
DON'T keep your temporary files in folders that will be backed up.