Login to ECMWF Cray

Each participant will have a training user account on the ECMWF Cray system. This is different from the user account on the classroom computers.

Begin by logging onto the Cray login nodes:

ssh trcrayXX@ccb                        <<< use your Cray user id
Password: xxxxxxxx

trcray@ccb-login2:~>

The contents of the account should look like:

trcray1@ccb-login2:~> ls
bin  modules.sh  oifs38r1  scratch

If any are missing please let us know.

Directory: oifs38r1

This directory contains the OpenIFS source code and OpenIFS executable.

For the purpose of this workshop, the model has already been compiled to avoid delay and adding unnecessary load to the login nodes.

Examine the files in the oifs38r1 directory.

Directory: scratch

This is the directory in which we will run OpenIFS.

Setting the correct environment

Each time you login and before doing any work in this account, make sure you have the correct environment loaded to run and work with OpenIFS.

source bin/modules.sh

This will load the appropriate modules for OpenIFS and set the choice of compiler and build environment. It is good practise to use something similar for your own OpenIFS installation.

Creating the OpenIFS ensemble experiment

An ensemble experiment consists of a 'control' and multiple 'perturbed' forecasts, or 'ensemble members'. For the operational ECMWF ensemble forecast, 50 members are used.

For this exercise, to save computer time, each participant will run a single, different 'ensemble member' and we will create the complete forecast ensemble from the individual ensembles run by the class as a whole. A later exercise will show how to do this from the model output GRIB files.

The first exercises show how to create the experiment directories, prepare the jobs and submit them to the ECMWF Cray.

createENS command

Use the command createENS to create the ensemble experiment:

createENS --help

usage: createENS [-h] -d <startdate/enddate/hr_inc> [-m <members>]
Set up ensemble forecast for the OpenIFS 2015 workshop
optional arguments:
  -h, --help            show this help message and exit
  -d <startdate/enddate/hr_inc>, --date <startdate/enddate/hr_inc>
                        List of date(s) to retrieve, format: YYYYMMDDHH. The
                        only dates available are: 2013102400/2013102700/24
  -m <members>, --members <members>
                        Number of ensemble members to create e.g. 3 (default 1)

Decide which of the available dates to use. Use the createENS command to create an ensemble with just a single member.

Change to the experiment directory labelled by the date chosen:

cd scratch/ENFO/2013102400/       #  your date may be different

There are two experiment directories: cf000 and pf001. cf000 is for the control forecast which is not used here. pf001 is the single perturbation forecast.

cd pf001
ls

Understand what the different files in the experiment directory are for.

ICM* : these are the initial starting files. ICMGG* are the initial gridpoint files, ICMSH* are the initial spectral fields.

Use the 'grib_ls' command to examine the initial files:

grib_ls ICMGGgc1dINIT
grib_ls ICMSHgc1dINIT

Creating the OpenIFS batch job

The namelist file: namelistfc

cp namelistfc fort.4

Edit namelist for your perturbation number.

 

Submit job: qsub jobN

Checking job status: qstatu