| Section | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
ECMWF HPCF
ECMWF operates a Cray based High Performance Computing Facility (HPCF). It consists of two identical Cray XC40 clusters with their own storage, but with equal access to the high performance working storage of the other cluster. This provides the benefit of having one very large system but the dual clusters add significantly to the resiliency of the system.
Each cluster has 20 cabinets of compute nodes and 13 cabinets of storage. The compute nodes contain two Intel Xeon "Broadwell" processors each with 18 cores. Four compute nodes sit on one blade, sixteen blades sit in a chassis and there are three chassis per frame. This gives a maximum of 192, or 6,912 processor cores per cabinet.
For more details about the ECMWF HPCF, please see: http://www.ecmwf.int/en/computing/our-facilities/supercomputer.
Login to ECMWF Cray High Performance Computing Facility
...
(HPCF)
Each participant will have a training user account on the ECMWF system. This is different from the user account on the classroom computers.
First login to the ECMWF gateway computer. From here you will login to the HPCF.
| Code Block |
|---|
ssh troifsXX@ecgb06.ecmwf.int # substitute your user id for XX
troifsXX@ecgb06.ecmwf.int's password: # give your password when prompted using the securID hardware token (one time password) |
Next login to the Cray login nodes:
| Code Block | ||
|---|---|---|
| ||
ssh ccb
troifs@ccb-login2:~> |
The contents of the account should look like:
| Code Block | ||
|---|---|---|
| ||
troifs1@ccb-login2:~> ls
bin make python t21test scratch |
If any are missing please let us know.
Directory make: contains the OpenIFS executable for version 40r1. For the purpose of this workshop, the model has already been compiled to avoid delay and adding unnecessary load to the login nodes.
Directory t21test: contains an example low resolution test run for OpenIFS.
Direcories bin and python: contains commands and script for use during the workshop.
Directory scratch: is the directory in which we will run OpenIFS and process the output files.
| Warning |
|---|
Please do not store large files in your home directory on the gateway or HPCF login. All OpenIFS experiments and large files should be stored under 'scratch'. The command |
Once logged in, to access the Cray HPCF:
...
Creating the experiment initial files
This section will explain how to copy and change the experiment id.
...
ob00/2015110100 to ob00/2015110100 etc.
Copy previous experiment
Before creating the experiment, we first need to create the initial data. If you plan to rerun the ob00 or clim experiments and use the initial data 'as-is' you can skip this step.
...
| Code Block | ||
|---|---|---|
| ||
cd inidir/ob50 cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/201511* . # this will be slow! |
Useful Unix commands
| Code Block | ||
|---|---|---|
| ||
rm -rf 2015110100 rm -rf 2015110[1-5]00 |
...