Versions Compared

Old Version 33

changes.mady.by.user Unknown User (nagc)

Saved on

compared with

New Version 34

changes.mady.by.user Unknown User (nagc)

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Each participant will have a training user account on the ECMWF Cray system. This is different from the user account on the classroom computers.

Begin by logging onto the Cray login nodes:

Code Block
languagebash
ssh trcrayXX@ccb                        <<< use your Cray user id
Password: xxxxxxxx

trcray@ccb-login2:~>

...

An ensemble experiment consists of a 'control' and multiple 'perturbed' forecasts, or 'ensemble members'. For the operational ECMWF ensemble forecast, 50 members are used.

...

The exercises show how to create the experiment directories, prepare the jobs and submit them to the ECMWF Cray.

...

Info
titleExercise

Please only use the 2013102400 start date to ensure we can create the single full ensemble from everyone's runs in the class.

Use the createENS command to create an ensemble with just a single member.

Info
titleExercise

Change to the experiment directory labelled by the date chosen:

Code Block
languagebash
titlee.g.
cd scratch/ENFO/2013102400/

There are two experiment directories: cf000 and pf001. cf000 is for the control forecast which is not used here. pf001 is the single perturbation forecast.

Code Block
languagebash
titleexamine files in the experiment directory
cd pf001
ls

Understand what the different files in the experiment directory are for.

ICM* : these are the initial starting files. ICMGG* are the initial gridpoint files, ICMSH* are the initial spectral fields.

Use the 'grib_ls' command to examine the initial files:

Code Block
grib_ls ICMGGoifsINIT
grib_ls ICMSHoifsINIT

The file: namelistfc is the model 'NAMELIST'. It contains a list of variable settings or 'switches' that control what the model does. These variables are grouped into separate fortran namelists.

...

Info

Make a copy of the namelist file and edit it to set the ensemble member value to the one you are given.

Make sure you are in the 'pf001' directory.

Code Block
cp namelistfc mynamelist

vi mynamelist        # or use your favourite text editor; emacs etc.

and make the following changes to the namelist variables:

Panel
bgColorwhite
titleBGColorwhite
titleSet these namelist variables:
CTYPE="pf",   ! change the type of forecast to 'perturbed' from 'control'

NENSFNB=2,    ! change the ensemble member number to your assigned value. 

LSTOPH_SPBS=true,  ! this enables the stochastic backscatter scheme in the model dynamics

LSPSDT=true,       ! this enables the stochastic scheme for the physics tendencies

The variable NENSFNB is important as this determines the random number seed used by the stochastic schemes, by altering the ensemble member value.

Code Block
languagebash
titleUse the 'diff' command to check your changes
diff mynamelist namelistfc
Note

This is different from how the ensemble would be run in practisepractice. Ensemble member '2' would be run in directory pf002, member number '3' in pf003 and so on. But to keep this exercise simple, everyone will use the same experiment directory name but with a different ensemble member number.

...

Create and submit Cray batch job

The next steps are to create and submit a batch job to run on the ECMWF Cray XC30 system.

To create the batch job, use the oifs_run command. This creates a small batch job file ready to submit.

...

Info
titleExercise

 Use the oifs_run command to create the job. You need to give a value to the '-e' and '-l' options. The other options can be left to their default values. The '-l' option is the name of the namelist file you edited above.

The command will create a create a file 'job1'.

Submit this job:

Code Block
qsub job1

Check the job status:

Code Block
qstatu

These forecasts complete in about 15mins.

...

Info
titleExercise

Using your favourite text editor, check this file to make sure the model has run correctly.  It's usually best to start from the bottom of the file and scroll up.

If you see:

Code Block
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
   OPENIFS JOB FAILED
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

the model has failed.

The output from each model run goes into a directory named according to this run number. e.g. the default is 'output1'. So job2 would write the output to 'output2' and so on.

If the model fails, there are 2 files to look at in the appropriate 'output' directory:

...