CLUSTER

Since March 2020, ABiMS has set up a new Slurm cluster in parallel with the current SGE cluster. Slurm is, like SGE, a job planner. It is more modern, but the major difference is that it requires you to manage RAM. This is currently our biggest problem with the "old" one. The memory required by jobs under SGE regularly exceeds the capacity of the node and causes major errors. Slurm forces users to declare the requested RAM and kills the job if the threshold is exceeded. This allows us to save on infrastructure maintenance. The first servers/nodes in this new cluster were initially 4 nodes of 256 Core-Threads for 256 GB of RAM. The nodes of the old cluster are added to the new computing infrastructure. Currently, there are more CPUs available on Slurm than on SGE. There is no official doc for the moment but as the new cluster is based on the one implemented for the IFB, we can easily direct you to the IFB docs :)

Request an account

To be allow to submit jobs on Slurm, a Slurm Account is required.
If you want to test this new cluster Slurm, let us know via the support.abims@sb-roscoff.fr

Login node

The login node is now slum0:

$ ssh login@ssh.sb-roscoff.fr

The documentation

Please have a look at the training material:

• Doc IFB: https://ifb-elixirfr.gitlab.io/cluster/doc/slurm_user_guide/
• Slides: https://ifb-elixirfr.gitlab.io/cluster/trainings/
• Video: https://asciinema.org/a/zZrSazw5Fh7YmpHvUfvVQnzNi
• Tutorial: https://ifb-elixirfr.gitlab.io/cluster/doc/tutorials/analysis_slurm

SGE2SLURM

Please check this training material: SGE2SLURM

• qsub is replaced by sbatch. There are subtleties in the syntax of the instructions to declare CPU numbers, ...
• qlogin disappears. If you want to run a fast calculation without sbatch, you just prefix your command with srun from the slurm0 server.

srun hostname # you can observe that you reach one of the cluster node
srun ln --help
srun --mem=8GB foobar -p yo

• Now you need to reserve the RAM for your jobs. By default, it is 2Gb. If the job exceeds the requested RAM, it is killed. It is restrictive but it protects the servers from memory overflow and therefore, we gain stability. It becomes an iterative process to properly estimate neither too high nor too low RAM consumption.

The Partitions and default values

Step 1: Find a client for the SFTP/SCP

Find a client for the SFTP/SCP protocol:

• CyberDuck
• WinSCP
• FileZilla (WARNING: you must pay attention to not install a web browser toolbar like My Search Dial)

Step 2: Log on the server

Log on our server with the following information

• Host : bioinfo.sb-roscoff.fr
• Username and password from your ABiMS account
• Port : 22
• Protocol : SFTP

Step 3: Upload your data on the server

1. You must change the remote directory: double-click on the path and change "/home" -> "/projet" (or another path)

   

2. Drag and Drop your files in the remote directory

RULES

• Work only in the /projet or /scratch spaces
• Respect this spaces structure (see Usage section),

SSH connection

You must connect on our server by ssh, through a terminal :

ssh -Y <login>@bioinfo.sb-roscoff.fr

/projet

cdprojet

Each /projet space must respect the following structure :

projet_utilisateur/
├── projet1
│   ├── input
│   ├── finalresult
│   ├── script
│   ├── archive
│  
└── projet2
    ├── input
    ├── finalresult
    ├── script
    ├── archive

The backup system works with inclusion. For the /projet spaces that are on the nz server, only the following directories are backuped!

• finalresult/
• script/
• archive(s)/

For the recent nz11 server, no backup has been yet implemented!

/scratch

cdscratch