Versions Compared

Old Version 35

changes.mady.by.user Marcus Koehler

Saved on

compared with

New Version 36

changes.mady.by.user Marcus Koehler

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Panel
bgColor#f0f0ff
titleTasks - Set OpenIFS environment
  1. Connect Carry out the tasks above to connect to ccb as before and start an interactive session
  2. Type the command:  cd perm/oifs43r3
  3. Type the command:  source ./oifs-config.ccb.sh

...

Panel
titleTasks - Examine grib files

Use the grib_ls and grib_dump commands to examine the contents of the ICM files.

...

Panel
titleTasks - Run the model as a single process

 Copy the file namelists to fort.4 and run the model with a single task and single thread by executing the job script:

Code Block
% cp namelists fort.4
% ./jobrun.ppn


The model will expect to find a file called fort.4 in the same directory as the executable. This script copies the executable from make/opt/bin.

...

The model writes its output to a several files.

NODE_001.01 contains the text output (WRITE/PRINT statements). The numbers refer to task number and thread number. Only output from the master task & thread is normally output but this can be changed for debugging purposes.

ICM*epc8+0000 is the model output in GRIB format split into 2 files; one for the gridpoint, the other for spectral fields. These contain only a few output variables in this test. This file is a mix of GRIB1 and GRIB2 messages. See the Documentation for how to process this output.

ifs.stat is a small file that prints the model steps, time taken for each step and a 'norm' measure. This file can be usually ignored but is useful for debugging.

Panel
titleTasks - Examine model output

Look at the output from the model in the NODE_001.01 file from this successful run. Note the output of the model namelists and the statistics printed at the end. IFS has very comprehensive logging output which is useful for debugging and understanding the model's performance.

Look at the grib output files using the grib commands.

...

Panel
titleTasks - Enable OpenMP

Edit the file 'job' run.ppn and change the line: export OMP_NUM_THREADSNTHREADS=1 to export OMP_NUM_THREADS=2 NTHREADS=4

Run ./job run.ppn as above.

Do the reported CPU times change?

Use the grib_ls command to look at grib output files - what do you notice?

...

If this works, look in the NODE_001.01 output file for the line:

Code Block
NUMBER OF THREADS                 24

to verify the model ran with 2 4 OpenMP threads.

Panel
titleTasks - Enable MPI

 Edit the file 'job' run.ppn and change OMP_NUM_THREADS NTHREADS back to 1.

Change the line: NPROC=1 to NPROC=24.

Also, edit the fort.4 file and change NPROC to 24.

Rerun the job:
     ./jobrun.ppn

Do the reported CPU times change?

Note that increasing the number of tasks requires changing the number of tasks in two places.

Look in the NODE_001.01 output file for the line: "NUMBER OF TASKS   2"  to verify that two MPI tasks was used.

...

The model can also be set to use NPROC=2 and OMP_NUM_THREADS=2 4 and NTHREADS=4 to use a total of 4 processes. However, this would require a computer with at least 4 16 cores.

Acceptance testing

The final step is to check the model is producing the numerical answers within acceptable limits, even if it runs the short tests above without failing. OpenIFS includes code that will compute internal statistical norms and compare against numbers supplied by ECMWF. The file: ref_021_0144 in the t21test directory contains statistical norms computed by the model run at ECMWF.

Panel
titleTask - run acceptance test

 To do the acceptance test, edit the namelists in fort.4 and look for the NAMCT0 namelist:

Code Block
&NAMCT0
 LREFOUT=false,
 NSTOP=6,

change the number of timesteps to 72 to run the model for 12 hours (assuming you have not changed the default timestep of 10mins at T21) and set the LREFOUT to TRUE:

Code Block
&NAMCT0
 LREFOUT=true,
 NSTOP=72,



With LREFOUT=true, at the last timestep OpenIFS will read the ref_021_0144 file and produce a new file: res_021_0144 (note the similar filenames!). The contents of the file should be similar to:

...

The namelist variables that determine the output from the model as it runs are:

Namelist : NAMCT0

LFPOS NFPOS - this should be set TRUE =2 in order to turn on model output and diagnostics.

...

               e.g
               NFP3DFS = 5,
               MFP3DFS = 130, 135, 138, 155,
               would output the temperature (130), vert. vel. vertical velocities (135), relative vorticity (138), divergence (155) on model levels.

...