Skip to end of metadata
Go to start of metadata

Post-processing of model outputs

  • mslp_${day}.grib for mean sea level pressure,
  • t2_${day}.grib for 2-meter temperature,
  • p_${day}.grib for precipitation,
  • gust_${day}.grib for 10-meter wind gust,
  • t850_${day}.grib for temperature at 850 hPa,
  • q700_${day}.grib for relative humidity at 700 hPa,
  • z500_${day}.grib for geopotential at 500 hPa,
  • u250_${day}.grib for horizontal wind components at 250 hPa,
  • u100_${day}.grib for horizontal wind components at 100 hPa,

where day is the given date in format of yyyymmdd (e.g., 20151203). All the files should contain data for 8 timesteps per day (i.e., in every 3 hours).

Please note the lowercase letters in the filenames.

The mean sea level pressure, the 2-meter temperature, the precipitation and the wind gust are expected in gridpoint representation. So they are prepared from the raw ICMGG* outputs with the next operations:


On this page


Useful links

% grib_copy -w shortName=msl  ICMGG${expID}+00${step} mslp_${day}_${step}.grib  #to get the mean sea level pressure
% grib_copy -w shortName=2t   ICMGG${expID}+00${step} t2_${day}_${step}.grib    #to get the 2-meter temperature
% grib_copy -w shortName=10fg ICMGG${expID}+00${step} gust_${day}_${step}.grib  #to get the 10-meter wind gust

where expID is the 4-digit experiment ID and step is the post-processing step (corresponding to every 3 hours) in 4 digits. For precipitation, both the convective and large-scale precipitation components have to be gathered in the same file:

% grib_copy -w shortName=lsp/cp ICMGG${expID}+00${step} p_${day}_${step}.grib

The pressure level data are required in spectral representation. So they are prepared from the raw ICMSH* outputs with the next operations:

% grib_copy -w shortName=t,level=850 ICMSH${expID}+00${step} t850_${day}_${step}.grib  #to get the temperature at 850 hPa
% grib_copy -w shortName=r,level=700 ICMSH${expID}+00${step} q700_${day}_${step}.grib  #to get the relative humidity at 700 hPa
% grib_copy -w shortName=z
,level=500 ICMSH${expID}+00${step} z500_${day}_${step}.grib  #to get the geopotential at 500 hPa

For wind, both the u and v components have to be collected in the same file:

% grib_copy -w shortName=u/v,level=250 ICMSH${expID}+00${step} u250_${day}_${step}.grib #to get the u and v components at 250 hPa
% grib_copy -w shortName=u/v,level=100 ICMSH${expID}+00${step} u100_${day}_${step}.grib #to get the u and v components at 100 hPa

After these operations, the timesteps belonging to the same days have to be merged into a common file:

% cat ${variable}_${day}_*.grib > ${variable}_${day}.grib

The size of the resulted files varies by the spatial resolution and the representation of the data. For instance, the file size at T255L91 resolution is 10 MB and 8 MB per variables for gridpoint and spectral fields, respectively, whereas these values increases to 35 MB and 26 MB at T639L137, to 233 MB and 179 MB at T1279L137.

Preparation of reference data

As reference data, we use the ECMWF re-analyses. Both ERA-Interim and ERA5 datasets are accessible in the ECMWF MARS (Meteorological Archival and Retrieval System). (Please note that the figures in the Case studies section are based on station observations which are however not publicly accessible.) Re-analyses are created by optimal combination of available observational information and short-range numerical weather predictions using data assimilation techniques. They provide the most comprehensive description of the past and current states of the 3-dimensional atmosphere or the Earth system.

ERA-Interim dataset (Dee et al., 2011) was prepared on 79 km horizontal resolution with 60 vertical levels starting from 1979. Analysis fields were constructed in every 6 hours using a variety of observations (conventional measurements, remote sensing data, extra space measurements etc.), the 4D-Var data assimilation technique and the IFS model version which was operational in 2009 (cycle 31r2). The forecasts initialized from the analysis produced 3 hourly outputs up to 24 hours.

ERA5 (Hersbach and Dee, 2016; Hersbach et al., 2018) is being constructed on higher, 32 km horizontal resolution with 137 vertical levels from 1950. Analysis fields are being prepared hourly with inclusion of newly reprocessed observational data, using the 4D-Var data assimilation technique and the IFS cycle 41r2 model version (operational in 2016). ERA5 forecasts initialized from the analyses at 6 and 18 UTC produce hourly outputs up to 18 hours and give an estimation of forecast uncertainty. There is an important difference between ERA-Interim and ERA5 in handling of the accumulated parameters: in ERA5 the accumulation is calculated from the previous post-processing step (i.e., along one hour), while in ERA-Interim it is from the beginning of the forecast – this feature is relevant in evaluation of the precipitation amount and wind gust. (More information about the characteristics of ERA-Interim and ERA5 can be found in the Copernicus Knowledge Base: What are the changes from ERA-Interim to ERA5?)

  • ${dataset}_mslp_${period}.grib for mean sea level pressure,
  • ${dataset}_t2_${period}.grib for 2-meter temperature,
  • ${dataset}_p_${period}.grib for total precipitation,
  • ${dataset}_gust_${period}.grib for 10-meter wind gust,
  • ${dataset}_t850_${period}.grib for temperature at 850 hPa,
  • ${dataset}_q700_${period}.grib for relative humidity at 700 hPa,
  • ${dataset}_z500_${period}.grib for geopotential at 500 hPa,
  • ${dataset}_u250_${period}.grib for horizontal wind components at 250 hPa,
  • ${dataset}_u100_${period}.grib for horizontal wind components at 100 hPa,

where dataset is a 2-digit identifier of the re-analysis data, ei for ERA-Interim and ea for ERA5; period is the investigated time period in format of yyyymmdd-yyyymmdd (e.g., 20151201-20151206 for Desmond).

Please note again the lowercase letters in the filenames. Furthermore, the re-analysis data should not be split by day, because data for the whole period will be handled together by the Metview macros.

The mean sea level pressure, the 2-meter temperature, the precipitation and the wind gust are expected in gridpoint representation, while the pressure level data are required in spectral representation. Total precipitation and wind gust as parameters representing a time period derive from forecasts, all the other variables are real analyses. Consequently, the daily quantities for precipitation and wind gust are composed of 8 and 24 timesteps from ERA-Interim and ERA5, respectively, the other variables have 4 and 8 timesteps (recall that output frequency of the forecast experiment is 3 hours). Besides the two (large-scale and convective) precipitation components, total precipitation is also available for direct retrieve both in ERA-Interim and ERA5, with GRIB code 228.

The necessary re-analysis data are available in the ECMWF download server in the proper format. For more information please visit http://download.ecmwf.int/test-data/openifs/reference_casestudies.

The re-analysis data can be retrieved directly from the MARS archive, too.

 Please click here for further instructions to retrieve ERA-Interim data.

The scr_download_re-analysis shell script is prepared to get the necessary data from MARS. The following steps have to be accomplished:

  1. Using the scr_download_re-analysis script requires Python 2.7 or higher version. Please make sure that the right Python version is installed on your computer.  
  2. ERA-Interim data are available in the ECMWF public datasets. To have access to the ECMWF public datasets, an account is needed to the ECMWF web site: https://apps.ecmwf.int/registration/.
  3. To retrieve data from MARS through ecmwfAPI, an ECMWF key is needed to be downloaded and installed. This page shows that step by step: Access ECMWF Public Datasets.
  4. Having the key, the scr_download_re-analysis shell script has to be run. It is available in the casestudies/mars directory in the OpenIFS cycle 40r1v2 release as well as in the ECMWF download server: http://download.ecmwf.int/test-data/openifs/reference_casestudies/programs/mars. It uses get_ecmars for retrieve which is also located both in the OpenIFS cycle 40r1v2 release (in the bin directory) and in the ECMWF download server.
  5. After submission of the retrieval, it can be checked in the Web-API job list (after login). There is also possibility here to cancel unwanted jobs.

The re-analysis source (i.e., ERA-Interim or ERA5), the surface and pressure level variables to be retrieved, the period of the data and the output directory have to be specified as options for the scr_download_re-analysis script. Calling it with -h option, it provides a detailed help with some examples at the end (but calling it without any option, it also gives a short instruction to its configuration):

% ./scr_download_re-analysis -h

 ----------------------------------------------------------------------
 This script downloads surface and pressure level ERA-Interim or ERA5 re-analysis data for a given time range from MARS.

 Usage:
        -c[class] -s[surface_variables] -p[plevel_variables] -f[firstdate:yyyymmdd] -l[lastdate:yyyymmdd] -o[output_directory]
        -h/-help

 Examples:
        ./scr_download_re-analysis -cei -s t2,p,mslp,gust -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o../reference
        ./scr_download_re-analysis -cea -s all -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o/home/rd/digs/metview/paper_OIFS/input/reference
        ./scr_download_re-analysis -ce5 -p t850 -f20151203 -o/home/rd/digs/metview/paper_OIFS/input/reference
  ----------------------------------------------------------------------

Please take into account that the script is able to retrieve only the variables listed above. For further parameters, the program has to be modified manually.

Furthermore, please note the all option which can be used with -s and -p switches. That takes automatically all the surface and/or pressure level variables discussed above (instead of listing them in the command line).

 Please click here for further instructions to retrieve ERA5 data.

The scr_download_re-analysis shell script is prepared to get the necessary data from MARS. The following steps have to be accomplished:

  1. Using the scr_download_re-analysis script requires Python 2.7 or higher version. Please make sure that the right Python version is installed on your computer.  
  2. Since March 2019, ERA5 data are available through Copernicus Climate Data Store (CDS). To have access to the data, an account is needed to CDS: https://cds.climate.copernicus.eu/user/register.
  3. You will need the CDS API client. It is a python based library, providing support for both Python 2.7.x and Python 3. You can install the CDS API client via the package management system pip, by running on Unix/Linux this command: % pip install cdsapi.
  4. To retrieve data through CDS API, a CDS key is needed to be downloaded and installed. This page shows that: How to use the CDS API.
  5. Before you may download any data from the CDS, you need to have accepted the Terms and Conditions of the chosen dataset. To do this:
    1. Go to the C3S Climate Data Store,
    2. On the top menu bar, click on Datasets,
    3. On the left-hand side menu, expand Product type and select Product type of interest (e.g. Reanalysis for ERA5 datasets),
    4. Follow the dataset title link of interest to the full dataset record,
    5. Accept dataset licence.
     
  6. Having the key, the scr_download_re-analysis shell script has to be run. It is available in the casestudies/mars directory in the OpenIFS cycle 40r1v2 release as well as in the ECMWF download server: http://download.ecmwf.int/test-data/openifs/reference_casestudies/programs/mars. It uses get_ecmars for retrieve which is also located both in the OpenIFS cycle 40r1v2 release (in the bin directory) and in the ECMWF download server.

The re-analysis source (i.e., ERA-Interim or ERA5), the surface and pressure level variables to be retrieved, the period of the data and the output directory have to be specified as options for the scr_download_re-analysis script. Calling it with -h option, it provides a detailed help with some examples at the end (but calling it without any option, it also gives a short instruction to its configuration):

% ./scr_download_re-analysis -h

 ----------------------------------------------------------------------
 This script downloads surface and pressure level ERA-Interim or ERA5 re-analysis data for a given time range from MARS.

 Usage:
        -c[class] -s[surface_variables] -p[plevel_variables] -f[firstdate:yyyymmdd] -l[lastdate:yyyymmdd] -o[output_directory]
        -h/-help

 Examples:
        ./scr_download_re-analysis -cei -s t2,p,mslp,gust -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o../reference
        ./scr_download_re-analysis -cea -s all -p t850,q700,z500,u250,u100 -f20151203 -l20151205 -o/home/rd/digs/metview/paper_OIFS/input/reference
        ./scr_download_re-analysis -ce5 -p t850 -f20151203 -o/home/rd/digs/metview/paper_OIFS/input/reference
  ----------------------------------------------------------------------

If you have other possibility to retrieve the ERA5 re-analysis data (e.g. via MARS client), please consider to use those one in order to avoid the sometimes long-lasting queues in CDS.

Please take into account that the script is able to retrieve only the variables listed above. For further parameters, the program has to be modified manually.

Furthermore, please note the all option which can be used with -s and -p switches. That takes automatically all the surface and/or pressure level variables discussed above (instead of listing them in the command line).

The retrieved ERA-Interim fields occupy approximately 54 MB, while the ERA5 fields take 700 MB for the Desmond case (i.e., for 1–6 December 2015).

References

Dee, D.P., Uppala, S.M., Simmons, A.J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M.A., Balsamo, G., Bauer, P., Bechtold, P., Beljaars, A.C., van de Berg, L., Bidlot, J., Bormann, N., Delsol, C., Dragani, R., Fuentes, M., Geer, A. J., Haimberger, L., Healy, S.B., Hersbach, H., Hólm, E.V., Isaksen, L., Kållberg, P., Köhler, M., Matricardi, M., McNally, A.P., Monge‐Sanz, B.M., Morcrette, J., Park, B., Peubey, C., de Rosnay, P., Tavolato, C., Thépaut, J., Vitart, F., 2011: The ERA‐Interim reanalysis: configuration and performance of the data assimilation system. Q.J.R. Meteorol. Soc. 137, 553–597. doi: 10.1002/qj.828

Hersbach, H., Dee, D.P., 2016: ERA5 reanalysis is in production. ECMWF Newsletter 147, p. 7.

Hersbach, H., de Rosnay, P., Bell, B., Schepers, D., Simmons, A., Soci, C., Abdalla, S., Alonso-Balmaseda, A., Balsamo, G., Bechtold, P., Berrisford, P., Bidlot, J-R., de Boisséson, E., Bonavita, M., Browne, P., Buizza, R., Dahlgren, P., Dee, D., Dragani, R., Diamantakis, M., Flemming, J., Forbes, R., Geer, A., Haiden, T., Hólm, E., Haimberger, L., Hogan, R., Horányi, A., Janisková, M., Laloyaux, P., Lopez, P., Muñoz-Sabater, J., Peubey, C., Radu, R., Richardson, D., Thépaut, J-N., Vitart, F., Yang, X., Zsótér, E., Zuo, H., 2018: Operational global reanalysis: progress, future directions and synergies with NWP. ECMWF ERA Report Series 27.


Back to top