Reference: HPC2020: How to connect

ssh hpc-login
hostname

ssh ecs-login
hostname

hpc-login hostname is an alias to a load-balanced service of login nodes. You may land on a different one every time you connect.

ecs-login is an alias to a specific login node of the ECS virtual cluster. It is not automatically load-balanced, so you will typically land on the same node on consecutive connections.

Both aliases will always point to a working login node, and the actual node and complex behind it may change depending on the load, system sessions or outages.

Password-less SSH between ECMWF hosts such as Atos HPCF or ECS nodes, or VDI hosts is not set up by default. If you were asked for a password, you can run the following command from your Atos HPCF, ECS or VDI session to set up  key-based authentication:

ssh-key-setup

After this you should be able to jump between hosts without having to introduce your password.

Besides being convenient, this setup is also necessary for other tools such as ECACCESS or ecinteractive to work properly.

Reference: HPC2020: Persistent interactive job with ecinteractive

ecinteractive -c 4 -m 10 -t 8:00

ecinteractive

ssh hpc-login
hostname

Reference: HPC2020: Filesystems

The default directory is your HOME directory, which is /home/$USER. It is a dedicated personal space for you, and you can always come back to that with either of the following commands:

cd
cd ~
cd $HOME
cd /home/$USER

Your HOME directory is accessible across all Atos HPCF, ECS, VDI and EcFlow services.

touch $PERM/del.me
touch $HPCPERM/del.me
touch $SCRATCH/del.me

ls -l $PERM/del.me
ls -l $HPCPERM/del.me
ls -l $SCRATCH/del.me

rm $PERM/del.me
rm $HPCPERM/del.me
rm $SCRATCH/del.me

quota

Quota for $HOME:
home_b             user    1234        <space used>   <space limit>       <number of files stored>       -   *

Quota for $PERM
POSIX User      1234    <space used>   <space limit>       <number of files stored>       none

Project quota for $SCRATCH and $SCRATCHDIR:
Disk quotas for prj 1000001798 (pid 1000001798):
     Filesystem    used   quota   limit   grace   files   quota   limit   grace
       /ec/res4     XXX     YYY     YYY       -     ZZZ     WWW     WWW       -

Project quota for $HPCPERM:
Disk quotas for prj 2000001798 (pid 2000001798):
     Filesystem    used   quota   limit   grace   files   quota   limit   grace
       /ec/res4     XXX     YYY     YYY       -     ZZZ     WWW     WWW       -

HOME and PERM are NFS-based Filesystems, which are mounted on all user computing platforms at ECMWF. You may access them with $HOME and $PERM environment variables:

ls $HOME
ls $PERM

However, SCRATCH and HPCPERM  are Lustre Based filesystem only available on the Atos HPCF, so  they are not available on other computing platforms such as VDI or ecFlow VMs and the corresponding environment variables are therefore not defined.

echo "hello world" > test_file.txt
ecp test_file.txt ec:
els -l ec:test_file.txt
ecp ec:test_file.txt retrieved_test_file.txt
diff test_file.txt retrieved_test_file.txt
erm ec:test_file.txt

touch $TMPDIR/testfile
touch $SCRATCHDIR/testfile
touch /tmp/testfile

ls -l $TMPDIR/testfile
ls -l $SCRATCHDIR/testfile
ls -l /tmp/testfile

HOME would be the preferred choice. They are typically small but important files, so convenience of backups, snapshots and availability on all computing platforms is more important than parallel performance. 

HPCPERM is the right choice for big files that are going to be used concurrently by parallel applications such as NWP models.

SCRATCH is the go to place for your daily work. Plenty of space, good parallel performance for output data that is transient by nature. Remember to move the data you want to keep after your job somewhere else, since files not used for 30 days will be automatically deleted.

PERM if accessibility from other computing platforms or the need of snapshots is important. You can see PERM as an extension to your HOME space.

HPCPERM, if I/O performance is more important than, especially if they are going to be used in parallel jobs on Atos HPCF.

ECFS would be the right place for longer term archival or storing backups. This is by far the place where you can store  However, data on tapes needs to be retrieved to another disk space before it can be used, so it is costly in terms of time.

In order to use ECFS efficiently, remember to store fewer but bigger files, so it is a good idea to use tools like tar or zip to bundle together big directories with lots of files.

$TMPDIR if performance is important and size is small, since TMPDIR is either in memory (for parallel jobs on HPCF), or on SSD disk.

$SCRATCHDIR if size of the files is big and does not fit TMPDIR.

You can use the snapshots . You can list all the versions available with:

ls -l ~/.snapshot/*/.profile

To recover, you would just need to copy the file back into place. 

For longer time spans, use the utility home_snap to get the locations

You can use the snapshots . You can list all the versions available with:

perm_snap

Note that the snapshots are less frequent in PERM.

Unfortunately there are no snapshots or backups for those filesystems, so the data has been lost permanently.