HPC changeover 2024¶
We will be changing our HPC, completing in May 2024. The new HPC, called Double Helix, will use Cloud compute, making it faster. This will change the way you connect to the HPC and call modules. You will receive an email when it is time for you to switch over.
This document reflects our current plans. We reserve the right to change these plans as things arise during the transition process.
What will change?¶
- Connect to the HPC: The address you use to connect to the HPC will change. If you have an ssh config file set up, you will need to change this.
- Call software modules: The module paths will be changing. We recommend that you run
module availto find the new paths for your favourite modules. You will need to change these in any workflows you have built.
- Pre-built workflows: We will bring out new versions of the workflows that use the new software module calls. If you have a copy of these in your own working directories, you will need to get a new copy.
What will stay the same?¶
- The filesystem: The filesystem and folder structure will not be changed. You do not need to move or copy your files to a new location. You can continue to write to your working folder before, during and after transition.
- The queues: We will keep the hpc job queues
inter, although we may add more queues.
- Tools available: We will make our best effort to install the same tools and modules on the new HPC, however this may not be possible with some older versions. The paths to call all tools will change; please run
module availto find the correct call.
What do I need to do?¶
You will receive an email informing you when it's time for you to switch to the new HPC. You should not run any jobs on the HPC while you are being switched over. Any jobs that you already have running will finish, but eventually your access to the old HPC will be revoked.
We are looking for people to test the new HPC and give us their feedback. You will get a chance to work on a fresh compute instance with minimal competition for resources. If you would like to test the new HPC, please get in touch with us via Service Desk, logging a HPC-related issue.
What will happen to my ongoing jobs on the HPC?
Any jobs that you start before you transition to the new HPC will finish running. As the filesystem is not changing, they will write to your working directory as normal.
You should not start any new jobs on the old HPC during transition. You can start new jobs on the new HPC as soon as you have transitioned.
I only use the desktop applications, will this affect me?
No. This change will only affect researchers who use the HPC via the command line.
Will I still be able to use the same tools for my analysis?
Mostly yes, we will try to make all the same tools available on the new HPC, however this may not be possible with some older versions of tools. However the paths to the tools is changing. We recommend running
module avail to find the new paths for your tools.
Will I still be able to access my data?
Yes, the new HPC will access the same folders and filesystem as before. You do not need to move or copy any data.
Will I be able to access Helix (the old HPC) after I have migrated to Double Helix (the new HPC)?
No. Your jobs will finish on Helix during your transition, but after transition you should not attempt to connect to Helix again and your access will be revoked.
Will I still be able to use tools I have installed myself?
You will need to reinstall any tools you have compiled and installed yourself.
Please reach out to us with any questions you have about the new HPC via Service Desk, tagging your tickets as "Double helix".