The filesystems available are HOME and PERM.
File System | Suitable for ... | Technology | Features | Quota |
|---|---|---|---|---|
| HOME | permanent files, e. g. .profile, utilities, sources, libraries, etc. | NFS | It is backed up. Limited I/O performance | See HPC2020: Filesystems |
| PERM | permanent files without the need for automated backups, smaller input files for serial or small processing, std output, etc. | NFS | It is backed up. Limited I/O performance | See HPC2020: Filesystems |
OneDrive | Sharing information with end user device and other ECMWF or external users | Microsoft | Cloud based. See How to configure and mount OneDrive - Linux Virtual Desktop VDI FAQ | 1TB (see OneDrive) |
Please be aware that any "local" filesystems, like /tmp, will be cleaned up every month when the virtual machine is recreated from the latest updated image. So do not use those to store any information that you may want to keep.
Environment variables
Those filesystems can be conveniently referenced from your session and scripts using the environment variables of the same name: $HOME, $PERM .
How much have I used?
You can check your current usage and limits with the "ecquota" command
New filesystem structure
You will notice your filesystems have now a flatter and simpler name structure. If you port any scripts or code that had paths to filesystems hardcoded from older platforms, please make sure you update them. Where possible, try and use the environment variables provided, which should work on both sides pointing to the right location in each case:
| Before | After |
|---|---|
| /home/group/user or /home/ms/group/user | /home/user |
| /perm/group/user or /perm/ms/group/user | /perm/user |
| - | /hpcperm/user |
| /scratch/group/user or /scratch/ms/group/user | /scratch/user |
4 Comments
Daniel Ali
Perhaps worth to mention that:
Samuel Hatfield
Hi, it says HOME and PERM are shared with the HPC. But the HOME and PERM on my Linux VDI do not appear to be the same as those on AA. The quotas for these filesystems are also different on the HPC2020 page (HPC2020: Filesystems). Is this because the Linux VDI/HPC synchronisation hasn't actually been done yet? Will the HOME and PERM on Linux VDI eventually have quotas of 100GB and 10TB respectively?
Samuel Hatfield
Presumably this is something to do with the fact that Linux VDIs are still hosted at Shinfield Park.
Daniel Ali
Hello. First of all, I understand your confusion.
You're right, the storage in use in the Shinfield VDI at the moment is a prototype, not the final one. It was shared with TEMS, but now that TEMS has been integrated with the rest of the Atos HPC in Bologna, there's a new storage there that we're not using in the VDI in Shinfield because that would imply mounting those filesystems over the WAN.
Once the VDI service has been deployed in Bologna, it will use the same $HOME and $PERM as the Atos.
As far as I know, the $HOME and $PERM in the Linux VDI in Shinfield won't be synchronised or copied automatically to Bologna, but we will provide a way to our users so they can do it themselves.