| Section | ||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Login to ECMWF Cray High Performance Computing Facility (HPCF)
Each group or participant will have a training user account on the ECMWF system, beginning with 'troifs'. This is different from the user account on the classroom computers.
...
| Warning |
|---|
Please do not store large files in your home directory on the gateway or HPCF login. All OpenIFS experiments and large files should be stored under 'scratch'. The command |
Creating the experiment initial files
This section will explain how to copy and change the experiment id.
...
You do not need to run these experiments, the forecasts from a 10 member ensemble are made available on the classroom computers.
Copy previous experiment initial files
Before creating the experiment, first create the initial data. If you plan to rerun the ob00 or clim experiments and use the initial data 'as-is' you can skip this step.
...
| Code Block | ||||
|---|---|---|---|---|
| ||||
cd inidir/ob01 cp -rL /perm/rd/openifs/oifs_workshop_2017/expts-inidata/ob00/2015110[1-3]* . # this will take some time. |
Change experiment id
OpenIFS forecasts are identified by an 'experiment id', a four letter string.
...
If you are using multiple dates, remember to change the experiment id for each date.
Initial files
The initial files containing 2D and 3D fields to start the forecast are contained in file that begin with ICM* : ICMGG* are the initial gridpoint files, ICMSH* are the initial spectral fields. ICMCL* is the file containing the climatological forcing fields.
...
| Code Block | ||
|---|---|---|
| ||
grib_ls ICMGGob00INIT grib_ls ICMSHob00INIT |
OpenIFS namelist
The file: fort.4 is the model 'NAMELIST'. It contains a list of variable settings or 'switches' that control what the model does. These variables are grouped into separate fortran namelists.
...
The wave model, WAM, also has a separate namelist. For this workshop, none of the switches in this file needs to be changed.
Listing the experiment id
The command 'exptid' can be used to check and change the experiment id in the GRIB files.
...
Note that the ICMCL and ICMGG*INIT files both have a experimentVersionNumber key of '0001'. This is used for climatological fields.
Set new experiment id
Use the exptid command to set the new experiment id to 'ob01':
...
| Code Block | ||
|---|---|---|
| ||
troifs0@cca-login3:> rm ICM*ob00* rm: remove regular file `ICMCLob00INIT'? y rm: remove regular file `ICMGGob00INIT'? y rm: remove regular file `ICMGGob00INIUA'? y rm: remove regular file `ICMSHob00INIT'? y |
Further modification to initial files
Now the initial files with the correct experiment ID have been prepared, further modifications to the input files can be made.
To modify the SST for example, see the tutorial on 'Modifying the SST'.
Creating the forecast experiment
Once the initial data directory is prepared, the next step is to create the experiment directory structure where the model will run. This is not the same location as the initial files just created.
...
This would create the ob01 forecast directories for a single date. The default is to create the experiment directory in your 'scratch' directory with the same name as the experiment id. e.g $HOME/scratch/ob01.
New experiment: ob01 - single data, 10 members
Following the creation of the initial files for the experiment id 'ob01' given above, let's create a forecast experiment with 5 ensemble members for a single forecast date:
...
| Code Block | ||
|---|---|---|
| ||
Creating directory structure for experiment ob01 in directory /scratch/ectrain/troifs0/ob01/... Date : 2015110100 Copying files from directory (inidir): scratch/inidir Date: 2015110100 ......... Created forecast experiment directory : /scratch/ectrain/troifs0/ob01/2015110100/05/ Linking files and copying namelist. Using IFS data directory: /fwsm/lb/project/openifs/ifsdata/ Created forecast experiment directory : /scratch/ectrain/troifs0/ob01/2015110100/06/ Linking files and copying namelist. Using IFS data directory: /fwsm/lb/project/openifs/ifsdata/ Created forecast experiment directory : /scratch/ectrain/troifs0/ob01/2015110100/07/ Linking files and copying namelist. Using IFS data directory: /fwsm/lb/project/openifs/ifsdata/ Created forecast experiment directory : /scratch/ectrain/troifs0/ob01/2015110100/08/ Linking files and copying namelist. Using IFS data directory: /fwsm/lb/project/openifs/ifsdata/ Created forecast experiment directory : /scratch/ectrain/troifs0/ob01/2015110100/09/ Linking files and copying namelist. Using IFS data directory: /fwsm/lb/project/openifs/ifsdata/ All done: /scratch/ectrain/troifs0/ob01/ ready. |
Directory structure
If we look at in the directory: /scratch/ectrain/troifs0/ob01/2015110100, we see:
...
Only the model namelists, fort.4 for the atmosphere, wam_namelist for the wave model, are individual to a forecast ensemble member directory.
Running the forecast experiment
The command 'oifs_run' is normally used to create and submit the batch job to run the model. This command has a large number of options. For this workshop, this command has been configured with the correct defaults for the T255 resolution seasonal length forecasts.
...
| Code Block | ||
|---|---|---|
| ||
......... Ensemble member: 05 60c60 < NENSFNB=5, --- > NENSFNB=0, ! Ensemble forecast number Running command: oifs_run -e ob01 -x 1 -q Copied /home/ectrain/troifs0/bin/cce-opt/master.exe to current directory Using existing namelist in : fort.4 OpenIFS job created: job1 Submitted job: 8430146.ccapar ...... |
Setting the ensemble member
The command 'run_all_ens' does this step, no action is needed. This section explains the use of the ensemble member number by IFS.
...
The only namelist variable that needs changing in these experiments is NENSFNB, which the run_all_ens command does for you.
Create and submit Cray batch job
This step is done by the run_all_ens command. No user action is needed. This section provide more information on the oifs_run command.
...
| Panel | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||||
The common arguments to use are: -e, -l, -f. Other options are available which can usually be left to their default values. Note! the -f argument, the length of the forecast can be specified in various units: for example d10 means '10 days'. |
Checking the model output
During the course of the model run, files with names like ICMGGob01+00000, ICMGGob01+00012 etc will appear in the ensemble member directory.
...
| Panel | ||||||
|---|---|---|---|---|---|---|
| ||||||
When the model completes the forecast successfully, the following files will be found in the output directory. ICMSH are the spectral fields, ICMGG are the gridpoint fields. ICMGGclim_10u ICMGGclim_ci ICMGGclim_lsp ICMGGclim_sd ICMGGclim_sst ICMGGclim_stl4 ICMGGclim_swvl3 ICMGGclim_tsr ICMSHclim_sp NODE.001_01 ICMGGclim_10v ICMGGclim_cp ICMGGclim_msl ICMGGclim_slhf ICMGGclim_stl1 ICMGGclim_str ICMGGclim_swvl4 ICMGGclim_ttr ICMSHclim_t ifs.stat ICMGGclim_2d ICMGGclim_e ICMGGclim_nsss ICMGGclim_sshf ICMGGclim_stl2 ICMGGclim_swvl1 ICMGGclim_tcc ICMSHclim_d ICMSHclim_vo oifs.log ICMGGclim_2t ICMGGclim_ewss ICMGGclim_q ICMGGclim_ssr ICMGGclim_stl3 ICMGGclim_swvl2 ICMGGclim_tp ICMSHclim_lnsp ICMSHclim_z |
| Anchor | ||||
|---|---|---|---|---|
|
The next step is to process the output of the successful run and create the monthly mean fields that can be transferred to ICTP for plotting using Metview.
...
This command only processes one date at a time. It needs to be run separately for multiple dates in the same experiment id.
Monthly mean files
After the oifs_to_mv command finishes, the monthly mean files can be found in the directory 'mmeans' inside the top level date directory. For example, for experiment id ob01, date 2015110100, the directory is: ob01/2015110100/mmeans.
Inside this directory, there is one file per parameter with 5 monthly averages: Dec, Jan, Feb, March and April, with filenames for example as: oif_ob01_u_20151101_00.grib.
Transfer to ICTP classroom PCs
The monthly mean files can be transferred to ICTP with the ftp command, to be used on the classroom PC with metview.
Useful Unix commands
| Code Block | ||
|---|---|---|
| ||
qsub job |
...