Page History

...

First of all, letLet's try explore how to connect to the computing services via SSH:ATOS HPCF or ECS.

Accessing a login node

1. Access the default login node of the ATOS HPCF or ECS and take note of what node you are in 

   Expand
   title Solution - HPCF
   No Format ssh hpc-login hostname

   
   

   Expand
   title Solution - ECS
   No Format ssh ecs-login hostname

   
   

2. Open a new tab in your terminal and connect again. Did you get the same hostname? Why is that?

   Expand
   title Answer
   hpc-login hostname is an alias to a load-balanced service of login nodes. You may land on a different one every time you connect. ecs-login is an alias to a specific login node of the ECS virtual cluster. It is not automatically load-balanced, so you will typically land on the same node on consecutive connections. Both aliases will always point to a working login node, and the actual node and complex behind it may change depending on the load, system sessions or outages.

   
   

3. Now, from your open SSH session on Atos HPCF or ECS, connect to the main login alias again. Did it ask for a password? Can you set your account up so jumps between hosts are done without a password?

   Expand
   title Answer
   Password-less SSH between ECMWF hosts such as Atos HPCF or ECS nodes, or VDI hosts is not set up by default. If you were asked for a password, you can run the following command from your Atos HPCF, ECS or VDI session to set up  key-based authentication: No Format ssh-key-setup After this you should be able to jump between hosts without having to introduce your password. Besides being convenient, this setup is also necessary for other tools such as ECACCESS or ecinteractive to work properly.

   
   

Interactive session

Standard sessions on login nodes do not guarantee access to dedicated resources such as cpus or memory, and strict limits on those are imposed. For interactive sessions with dedicated resources we will use ecinteractive. 

It is installed and available on all the Atos HPCF and ECS nodes, as well as the VDI. If you are connecting from your own computer with teleport, you can download it and run it there (only Mac, Linux or Windows WSL are supported).

1. Can you get a dedicated interactive session with 10 GB of memory and 4 cpus for 8 hours?

   Expand
   title Solution
   You can use ecinteractive . It is installed and available on all the Atos HPCF and ECS nodes, as well as the VDI, so you can run it from therelike so: No Format ecinteractive -c 4 -m 10 -t 8:00 This will create an interactive job with the requested configuration and land you on a shell in a given node.If you are connecting from your own computer via teleport, you can download it and run it there (no Windows native support, only Mac, Linux or WSL supported). Note that by default and unless ran directly on ECS, ecinteractive will use HPCF as the backend. If you wish to force the session to be on ECS, you can dopass the extra option -p ecs No Format ecinteractive -p ecs -c 4 -m 10 -t 8:00

   
   

2. Log out of that interactive session. Can you reattach to it?

   Expand
   title Solution
   Your job kept running in the background, and there can only be one interactive job per user. You can attach as many concurrent shells to the same interactive session, for example in different terminal tabs, with: No Format ecinteractive If on ECS, you may want need to run: No Format ecinteractive -p ecs

   
   

3. How can you get information (amount of resources, time left, etc) of your active ecinteractive session?

   Expand
   title Solution
   At any point, from any session, you can query ecinteractive for any active sessions with: No Format ecinteractive -q If on ECS, you may want need to run: No Format ecinteractive -p ecs -q

   
   

4. Cancel your interactive session

   Expand
   title Solution
   No Format ecinteractive -k If on ECS, you may want need to run: No Format ecinteractive -p ecs -k

   
   

5. We can also use ecinteractive to open a JupyterLab instance on the Atos HPCF or ECS. Start a JupyterLab session with 4 GB of memory. When you are happy, cancel the ecinteractive job, since only closing the browser tab will not stop the session.

   Tip
   title Where to run it
   You must run ecinteractive from either VDI or your own computer. If you run the command on the Atos HPCF or ECS, it will not be able to open any browser window.

   
   

   Expand
   title Solution
   From VDI or your own computer, you can start up jupyter like so No Format ecinteractive -j -m 4 If you want to start it on ECS, you will need to do No Format ecinteractive -j -m 4 -p ecs No Format ecinteractive -k If on ECS, you may want need to run: No Format ecinteractive -p ecs -lk If you still have the JupiterLab tab open in your browser, you will notice it will stop working.