Introduction
The ERA5 family daily statistics catalogue entries provide post-processed daily aggregated data (for four statistics) for the ERA5 and ERA5-Land hourly data. These entries replace the ERA5 daily statistics application in the legacy Climate Data Store. This gives users a more consistent feel to accessing the data, and means that the returned data is more inline with other data files produced by the CDS.
The daily statistics are calculated as part of the retrieval and the data are not permanently archived. The daily statistics are calculated using the aggregate submodule in the earthkit-transforms python package. However, to ensure that the daily statistics calculated represent the period requested (e.g. for accumulated variables), several additional steps are taken, as documented here.
How the daily statistics are calculated
This section describes in details the process for calculating the daily statistics. Most users will only need to use the CDS webforms or the CDS API to access the data.
For a full technical demonstration of the calculation, please view the Jupyter Notebook below. The general workflow is:
- Initialise the request and map all the variables, including:
- the time zone selection is converted to a
time_shiftinteger which represents the number of hours. - The
frequencyselected is converted to an integer representing the number of hours
- the time zone selection is converted to a
- Loop over the requested variables:
- If the variable is an accumulated or mean-rate variable, we subtract one hour from the
time_shift- For ERA5 single-levels and ERA5 pressure-levels, the accumulated and mean rate variables represent the hour to the time-stamp, that is, the data time-stamped as YYYY/MM/DD 00:00, represents the accumulation/mean-rate of the data for the time period 23:00 to 00:00 for the date YYYY/MM/DD-1.
- If the time_shift is greater than zero, the preceding day is added to the request
- To ensure a full sampled period, only the UTC time-zone or time-zones West of UTC (i.e. UTC-HH:00) can be retrieved for the first available day of date (01/01/1940 for ERA5 and 01/01/1950 for ERA5-Land)
- If the time_shift is less than zero, the following day is added to the request
- The time steps required for the daily statistics calculation are created as a list and added to the request:
- time steps list
this_time: list[str] = [f"{i+(this_hour%frequency):02d}:00:00" for i in range(0, 24, frequency)]
- The data is requested and returned in grib format
- The data is opened using xarray with args and kwargs used by the parent entry.
- The daily statistic is calculated using the following command:
- time steps list
daily_data = earthkit.transforms.aggregate.daily_reduce(how=HOW, time_shift = {"hours": this_hour}, remove_partial_periods= True) # Where: # daily_mean, HOW="mean" # daily_max, HOW="max" # daily_min, HOW="min" # daily_sum, HOW="sum"
- The xarray objects is written to netCDF
- If the variable is an accumulated or mean-rate variable, we subtract one hour from the
- The netCDF file[s] are returned to the user
Jupyter notebook demonstrating the calculation of the daily statistics
Data organisation and access
The following table provides an overview of which daily data are available.
Table 1:
| Instantaneous parameters | Accumulated parameters | |
|---|---|---|
ERA5 single levels (reanalysis and ensemble) | ✔ | ✔ |
ERA pressure levels (reanalysis and ensemble) | ✔ | ✔ |
| ERA5-Land | ✔ | X |
The data are available from the Climate Data Store (CDS) webforms:
- Derived ERA5 daily statistics on single levels from 1940 to present
- Derived ERA5 daily statistics on pressure levels from 1940 to present
- Derived ERA5-Land daily statistics from 1950 to present
or programatically from the CDS API:
Spatial grid
The ERA5 reanalysis atmospheric data resolution is 0.25° and the ERA5 ensemble atmospheric data resolution is 0.5°.
The ERA5 wave data resolution is 0.5° and the ERA5 ensemble wave data resolution is 1.0°.
The ERA5-Land data resolution is on 0.1°.
Temporal frequency
Users can select to calculate daily statistics from 1 hour, 3 hours and 6 hours data.
Data format
The post-processed daily statistics are provided in netCDF format only (in a zip file). The structure and naming conventions used are the same as the hourly dataset used as input. The data is provided as one netCDF file per variable, and all files will be archived in a zip file for downloading.
Accumulated variables for ERA5 land
Please note that ERA5-Land daily accumulated parameters are not available from the catalogue entry.
Please, note that the convention for accumulations used in ERA5-Land differs with that for ERA5. The accumulations in the short forecasts of ERA5-Land (with hourly steps from 01 to 24) are treated the same as those in ERA-Interim or ERA-Interim/Land, i.e., they are accumulated from the beginning of the forecast to the end of the forecast step. For example, runoff at day=D, step=12 will provide runoff accumulated from day=D, time=0 to day=D, time=12. The maximum accumulation is over 24 hours, i.e., from day=D, time=0 to day=D+1,time=0 (step=24).
- HRES: accumulations are from 00 UTC to the hour ending at the forecast step
- For the CDS time, or validity time, of 00 UTC, the accumulations are over the 24 hours ending at 00 UTC i.e. the accumulation is during the previous day
The data time-stamped YYYY/MM/DD 00:00 represents the total daily accumulation for the date YYYY/MM/DD-1. Therefore:
- To calculate the daily accumulation for UTC, you just need to sample the ERA5-Land data at 00:00 and be aware that the data is representative of the day before the time stamp in the data
- Note, this is why we do not include this, as it would in effect be the same data but with a different time stamp, leading to confusion
- To calculate the daily accumulation for non-UTC time-zones we must sample the data at 2 time steps and then carefully combine them so that they are associated to the correct. This is quite complex and we provide the following notebook as guidance:
