Logging in - Atos HPCF and ECS Introduction Tutorial

Let's explore how to connect to the ATOS HPCF or ECS.

Accessing a login node

1. Access the default login node of the ATOS HPCF or ECS and take note of what node you are in 

   Solution - HPCF

   ssh hpc-login
   hostname

   Solution - ECS

   ssh ecs-login
   hostname

2. Open a new tab in your terminal and connect again. Did you get the same hostname? Why is that?

   Answer

   hpc-login hostname is an alias to a load-balanced service of login nodes. You may land on a different one every time you connect.

   ecs-login is an alias to a specific login node of the ECS virtual cluster. It is not automatically load-balanced, so you will typically land on the same node on consecutive connections.

   Both aliases will always point to a working login node, and the actual node and complex behind it may change depending on the load, system sessions or outages.

3. Now, from your open SSH session on Atos HPCF or ECS, connect to the main login alias again. Did it ask for a password? Can you set your account up so jumps between hosts are done without a password?

   Answer

   Password-less SSH between ECMWF hosts such as Atos HPCF or ECS nodes, or VDI hosts is not set up by default. If you were asked for a password, you can run the following command from your Atos HPCF, ECS or VDI session to set up  key-based authentication:

   ssh-key-setup

   After this you should be able to jump between hosts without having to introduce your password.

   Besides being convenient, this setup is also necessary for other tools such as ECACCESS or ecinteractive to work properly.
   

Interactive session

Standard sessions on login nodes do not guarantee access to dedicated resources such as cpus or memory, and strict limits on those are imposed. For interactive sessions with dedicated resources we will use ecinteractive. 

It is installed and available on all the Atos HPCF and ECS nodes, as well as the VDI. If you are connecting from your own computer with teleport, you can download it and run it there (only Mac, Linux or Windows WSL are supported).

1. Can you get a dedicated interactive session with 10 GB of memory and 4 cpus for 8 hours?

   Solution

   You can use ecinteractive like so:

   ecinteractive -c 4 -m 10 -t 8:00

   This will create an interactive job with the requested configuration and land you on a shell in a given node.

   Note that by default and unless ran directly on ECS, ecinteractive will use HPCF as the backend. If you wish to force the session to be on ECS, you can pass the extra option -p ecs

   ecinteractive -p ecs -c 4 -m 10 -t 8:00

2. Log out of that interactive session. Can you reattach to it?

   Solution

   Your job kept running in the background, and there can only be one interactive job per user. You can attach as many concurrent shells to the same interactive session, for example in different terminal tabs, with:

   ecinteractive

   If on ECS, you may want need to run:

   ecinteractive -p ecs

3. How can you get information (amount of resources, time left, etc) of your active ecinteractive session?

   Solution

   At any point, from any session, you can query ecinteractive for any active sessions with:

   ecinteractive -q

   If on ECS, you may want need to run:

   ecinteractive -p ecs -q

4. Cancel your interactive session

   Solution

   ecinteractive -k

   If on ECS, you may want need to run:

   ecinteractive -p ecs -k

5. We can also use ecinteractive to open a JupyterLab instance on the Atos HPCF or ECS. Start a JupyterLab session with 4 GB of memory. When you are happy, cancel the ecinteractive job, since only closing the browser tab will not stop the session.

   Where to run it

   You must run ecinteractive from either VDI or your own computer. If you run the command on the Atos HPCF or ECS, it will not be able to open any browser window.

   Solution

   From VDI or your own computer, you can start up jupyter like so

   ecinteractive -j -m 4

   If you want to start it on ECS, you will need to do

   ecinteractive -j -m 4 -p ecs

   ecinteractive -k

   If on ECS, you may want need to run:

   ecinteractive -p ecs -k

   If you still have the JupiterLab tab open in your browser, you will notice it will stop working.