Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Panel
titleBGColorlightgrey
titleCompiling OpenIFS / FCM related questions


Expand
titleQ. The command 'fcm make' gives the error: mpif90: command not found?
Expand
titleClick here for answer...

A. This is because the FCM configuration files assume the use of a 'compiler wrapper' which is typically provided by MPI (e.g. OpenMPI or MPICH) installations. Make sure you have a flavour of MPI installed and you have the 'bin' directory included on your PATH environment variable. For example, on linux use 'apt-get install libopenmpi-dev' to get OpenMPI. HPC machines may require you to use a 'module' command e.g 'module load openmpi': check your local help information.

Expand
titleQ. I get this error, what does it mean? "[FAIL] cpg1c.intfb.h: missing dependency (type=include)"
Expand
titleClick here for answer...

A. FCM has found an include statement: "#include cpg1c.intfb.h" in the source code but can't find the source file cpg1c.F90. Make sure the file exists or FCM is configured to look at all the sources.

Expand
titleQ. How do I change where FCM builds the library archive (libmaster.a) file before the link stage?
Expand
titleClick here for answer...

A. FCM queries the TMPDIR environment variable. If set it uses that location, otherwise it defaults to /tmp. On HPC systems, /tmp may not be the best choice if it is in memory.

Expand
titleQ. What does the $HERE variable in FCM configuration files mean?
expand
titleClick here for answer...

A. The $HERE variable is a special variable that FCM interprets as the directory in which the configuration file resides. Unlike most of the other variables in the FCM configuration files, $HERE cannot be overridden by an environment variable of the same name.name.


Expand
titleQ. How do I change the compiler used by my MPI compiler wrappers?
Expand
titleClick here for answer...

A. For more information on how to correctly set your environment for compiling OpenIFS, see: HowTos: How to set your user environment.

It varies depending on your MPI implementation and your local environment. As an example, for OpenMPI, set the environment variables:
export OMPI_FC=pgf90
export OMPI_CC=pgcc
to use the Portland compilers instead of the default GNU (gfortran) compilers. Check the 'man page' of the mpif90 command to see what other options are possible. Some implementations allow you to use an option -fc to set the underlying compilers.

For Intel installations there are several ways in which the compiler can be set. Normally, the mpif90 command will invoke the gfortran compiler, the mpiifort will invoke the ifort compiler and the mpifc command can be used with a specified compiler on the command line. However, this behaviour can be changed by setting environment variables which these wrapper scripts will read. For more details consult the Intel compiler documentation.

Some HPC centres provide the 'module' command for loading and unloading particular compiler suites. Consult your local documentation for more details.

Expand
titleQ. I get an error about missing shared object file: error while loading shared libraries: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
.
Expand
titleClick here for answer...

A. This can arise with the Intel compiler. The MKL library is Intel's version of the LAPACK and BLAS libraries which should be supplied with the compiler. The .so file extension means this is a shared library found at runtime. In order to find this library, the environment variable LD_LIBRARY_PATH must be set correctly. You will need to find the location of this library and add the directory to the LD_LIBRARY_PATH. Normally this is done for you by a small script supplied with the Intel compiler script. Please check your support information.

Expand
titleQ. I get undefined references to 'dgemm_', 'dgemv_', 'sgecon_' etc
.
Expand
titleClick here for answer...

A. These routines come from the LAPACK and BLAS libraries. You may have one or other of these libraries missing from your .cfg config (or OIFS_LAPACK_LIB environment variable). Alternately you have specific both libraries but the compiler can't find them; check the pathnames to the libraries specified on the compile link carefully.


Expand
titleQ. I get undefined references to: 'jas_stream_memopen', 'jpc_decode', 'jas_matrix_create', 'jas_image_destroy', 'jpc_decode'
Expand
titleClick here for answer...

A. These are references to the JASPER library and arise because the grib_api library referred to in the OpenIFS build configuration was compiled with the Jasper library, but the LD_LIBRARY_PATH environment variable does not list the directory containing the library so the compiler can't find it. Either amend the LD_LIBRARY_PATH or add the -ljasper library to your compilation options.


Expand
titleQ. Should I use the same compiler for OpenIFS, LAPACK, OpenMPI etc?
Expand
titleClick here for answer...

A. It's recommended that the same compiler suite is used to compile OpenIFS, your MPI installation and LAPACK. This may not always be practical but it's strongly recommended as some compilers use a different default size of LOGICAL variables which can cause crashes due to memory alignment problems in subroutine calls.


Expand
titleQ. Where is the log file from compiling OpenIFS kept?
Expand
titleClick here for answer...

A. The log output from compiling OpenIFS can be found in the 'make' directory, then in the build directories 'opt', 'noopt' etc. The FCM software keeps the log in a hidden directory called '.fcm-make'. So if OIFS_BUILD=opt was set, the logfile for this compilation would be : make/opt/.fcm-make/log.

...

Panel
titleBGColorlightgrey
titleOpenIFS questions: general and runtime


Expand
titleQ. What does the 'T' mean in 'T511', 'T1279' etc?
Expand
titleClick here for answer...

A. The 'T' refers to the spectral truncation of the model and the number following is the highest retained wavenumber. A spectral method, based on spherical harmonic expansion is used in the model, for example to compute some horizontal derivatives. The 'T' means the model uses a 'triangular' truncation in spectral space to decide the retained spherical harmonics. The number following the 'T' is the highest retained wave number. For more information on the spectral truncation and resolution of the model and the model grid, the IFS documentation is recommended reading.

T159 is approximately 125km resolution, T255 - 80km, T511 - 40km, T799 - 25km, T1279 - 16km.

Expand
titleQ. How do I know the grid from from the
'T' number?
Expand
titleClick here for answer...'T' number?

A. This depends on the grid used. For medium and high resolutions, the model uses a 'reduced Gaussian grid' with reduced numbers of gridpoints along a latitude circle towards the poles. Full quadratic and linear grids are also possible. This table shows the correspondence between spectral resolution and lat/lon resolution. The grid definitions can also be found here: reduced gaussian grid definitions.


Expand
titleQ. What is the difference between IFS/OpenIFS model versions?
Expand
titleClick here for answer...

A. The OpenIFS model is based on the main IFS code though is only capable of forecasts. The first OpenIFS version was based on cycle 38 release 1 (known as CY38R1). More information is found on the page comparing OpenIFS .v. IFS.

Information on scientific & technical changes between cycle releases is available on the main ECMWF website.


Expand
titleQ. What format is the model output?
Expand
titleClick here for answer...

A. The model writes its output in GRIB format using WMO FM-92 GRIB edition 1 and edition 2 messages. ECMWF provides the GRIB_API software for handling and manipulating grib data using either command line tools and a programming library for C, Fortran and Python.

For plotting GRIB data, ECMWF provides the MetView software, which is freely available under the Apache license.

Note that the OpenIFS model outputs a mix of GRIB edition 1 & 2 messages in a single file. Unlike MetView, not all GRIB software available on the Internet can read such files and extra steps may be necessary to separate the messages before use if you want to use 3rd party software.


Expand
titleQ. How do I check the model is working correctly on my system after I've installed it?
Expand
titleClick here for answer...

A. The model can be run in 'reference' mode where it computes statistical numerical norms and compares against values calculated at ECMWF. Please see 'Testing the Installation' for more details.


Expand
titleQ. What timestep should I use?
Expand
titleClick here for answer...

A. The timestep depends on the horizontal and vertical resolution, the type of grid (full or reduced Gaussian), whether Eulerian or semi-Lagrangian (SL) transport is used and whether the 2 or 3 time-level SL scheme is used. The recommended default is the 2 time-level SL scheme. For this configuration suggested timesteps are: T21 - T159 : 60mins, T213 -T319 : 20mins, T511 and above : 10mins. If the 2 time-level scheme is not used, these timesteps should be halved. If Eulerian, not SL transport is used, these timesteps should be a quarter. For more details, see the code in the routine sudyn.F90 which has approximate values for the timestep (n.b. the NAMDYN namelist variable TSTEP is in units of secs).

Note the model will calculate the optimum timestep for its particular configuration. This value is written to the NODE_001.01 output file. Look for the line:
PTSTEP_OPT (optimal timestep computed in SUTSTEP) 0.36000000E+04
to see the recommended value in seconds.


Expand
titleQ. OpenIFS fails with
 ABORT
ABORT!
   
1 SU_MCICA:ERROR OPENING FILE MCICA
Expand
titleClick here for answer...

A. The file 'MCICA' is a climatological input file the model needs. The model expects to read the climatological files from a sub-directory called 'ifsdata' in the directory containing the executable. Normally this is a link to where you installed the ifs data files. See Downloading OpenIFS for more details on how to download and install the data files.


Expand
titleQ. OpenIFS fails with an error message from GRIB_API
.
Expand
titleClick here for answer...

A. This usually indicates an error in the GRIB file or GRIB_API configuration rather than the model. See the GRIB FAQ on this page, or email openifs-support@ecmwf.int and include the model output file (NODE_001.01) and the job log.


Expand
titleQ. OpenIFS fails (segmentation violation) when compiled with optimized compiler options ('opt' configuration) but works without any optimization ('noopt' configuration)
.
Expand
titleClick here for answer...

A. This could be for a number of reasons. However, a common one is that when compiled optimized the model requires more memory.  With the optimized settings OpenMP is enabled. This causes the model to require more what's known as 'stack memory'. To increase your stack memory to the maximum allowed on your system, use the command 'ulimit -s unlimited'. If that does not solve your problem, email openifs-support@ecmwf.int.

Q. OpenIFS fails with error: Dr.Hook has detected an invalid key-pointer/handle while leaving the routine 'RPASSF'

Expand
titleClick here for answer...

A. This normally means that the OpenMP stacksize is not large enough. The default per process stack size limit is too low and needs to be increased. Check the value of the OMP_STACKSIZE environment variable.

Q. I don't get any output from the model?

Expand
titleClick here for answer...

A. This could be for a number of reasons. One possibility is that the model is configured as if running at ECMWF where it writes file to a special database. Check that the NODE_001.01 output file and make sure that in namelist NAMCT0, the variable LFDBOP is .FALSE. and in namelist NAMPAR1, the variable NOUTTYPE=1. If this is the ok, then please see our HowTo on changing the model output or contact openifs-support@ecmwf.int.

...