1. Context.The concept of "nominal" start date in lagged start ensembles
Within the seasonal forecast datasets available at C3S, we have contributions from two families of forecasting systems, depending on what techniques are used to build the members of their ensemble forecast, so that they sample the uncertainty in the initial conditions:
- "Burst" mode: all the members are initialized with conditions on the same start date, but from slightly different (perturbed) initial states, intended to sample the uncertainty in observations. (e.g. all members initialized on 1st March 2017; this is the case for ECMWF's system)
- "Lagged" mode: members are initialized on different start dates, the differences between which are sufficiently small (e.g. members initialized every day of the month; this is the case for Met Office system)
Due to a number of reasons, members from ensembles using the "lagged start" approach are bundled together at C3S and labelled with a "nominal" start date (see details here).
The C3S seasonal forecast datasets are based on GRIB files stored at ECMWF MARS archive where that "nominal" start date is used to index the monthly data (monthly statistics and monthly post-processed products like anomalies). Hence, the details about the original initialization date could get lost in those files. Nonetheless, the data at daily and sub-daily time frequencies is always encoded with the real initialization date, as it can be seen in the example below:
| Daily/Sub-daily data | Monthly data | ||
| Date | Ensemble member | Date | Ensemble member |
19931127 | 0 | 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 | 0 |
2. The encoding in GRIB format using "Local Definitions"
In December 2019, with the release of ecCodes 2.15.0, an extension to the GRIB encoding was introduced to allow storing additionally the original initialization date on those monthly files, while keeping the "nominal" start date as indexing date to be used by the MARS archive. The solution adopted was to create a new GRIB "local definition" for the seasonal forecast monthly data produced by lagged start systems.
Up to that date, all seasonal forecast monthly data files produced at ECMWF, and that includes the data provided by all C3S seasonal forecast contributors, was encoded using GRIB local definitions described in table 16 (GRIB keyword localDefinitionNumber=16 ). With the changes introduced with version 2.15.0 of ecCodes a new local definition has been devised for lagged start systems (GRIB keyword localDefinitionNumber=12) and its use has been implemented to all the lagged start systems data produced after December 2019.
Sytems affected by this change
- MetOffice GloSea5 (system=14) used
localDefinitionNumber=16up to December 2019 nominal start date, and started usinglocalDefinitionNumber=12from January 2020 onwards. (Note this affects not just the forecasts but also the relevant hindcasts) - NCEP CFSv2 uses
localDefinitionNumber=12since its introduction into the C3S multi-system (Note that this was achieved by delaying the publication of all monthly data until December 2019)
In the future, systems using the lagged start approach introduced into C3S will also use localDefinitionNumber=12
What has changed?
| localDefinitionNumber=12 (after December 2019) | |
|---|---|---|
| Date used in the CDS requests | Nominal start date GRIB keywords: dataDate, dataTime | Nominal start date GRIB keywords: indexingDate, indexingTime |
| Real start date | Not available | GRIB keywords: dataDate, dataTime |
It is important to bear in mind that most common grib tools will read dataDate and dataTime
It is therefore recommended to use tools based on ecCodes (version 2.15.0 or greater), specifically cfgrib (version 0.9.7.4 or greater) which can be used within python's xarray
3. Examples
Example 1: UK MetOffice hindcast data for December 1993 (localDefinitionNumber=16)
import cdsapi
c = cdsapi.Client()
c.retrieve(
'seasonal-monthly-single-levels',
{
'format': 'grib',
'originating_centre': 'ukmo',
'system': '14',
'variable': '2m_temperature',
'product_type': 'monthly_mean',
'year': '1993',
'month': '12',
'leadtime_month': [ '1', '2', ],
},
'c3s_seas_ukmo199312_t2m.grib')
> grib_ls -p origin,type,shortName,number,indexingDate,indexingTime,dataDate,dataTime,step,fcmonth,verifyingMonth c3s_seas_ukmo199312_t2m.grib c3s_seas_ukmo199312_t2m.grib origin type shortName number indexingDate indexingTime dataDate dataTime step fcmonth verifyingMonth egrr fcmean 2t 0 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 1 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 2 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 3 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 4 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 5 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 6 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 7 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 8 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 9 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 10 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 11 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 12 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 13 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 14 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 15 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 16 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 17 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 18 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 19 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 20 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 21 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 22 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 23 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 24 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 25 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 26 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 27 not_found not_found 19931201 0 744 1 199312 egrr fcmean 2t 0 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 1 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 2 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 3 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 4 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 5 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 6 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 7 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 8 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 9 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 10 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 11 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 12 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 13 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 14 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 15 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 16 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 17 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 18 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 19 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 20 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 21 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 22 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 23 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 24 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 25 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 26 not_found not_found 19931201 0 1488 2 199401 egrr fcmean 2t 27 not_found not_found 19931201 0 1488 2 199401
Example 2: NCEP hindcast data for December 1993 (localDefinitionNumber=12)
import cdsapi
c = cdsapi.Client()
c.retrieve(
'seasonal-monthly-single-levels',
{
'format': 'grib',
'originating_centre': 'ncep',
'system': '2',
'variable': '2m_temperature',
'product_type': 'monthly_mean',
'year': '1993',
'month': '12',
'leadtime_month': [ '1', '2', ],
},
'c3s_seas_ukmo199312_t2m.grib')
> grib_ls -p origin,type,shortName,number,indexingDate,indexingTime,dataDate,dataTime,step,fcmonth,verifyingMonth c3s_seas_ncep199312_t2m.grib c3s_seas_ncep199312_t2m.grib origin type shortName number indexingDate indexingTime dataDate dataTime step fcmonth verifyingMonth kwbc fcmean 2t 0 19931201 0 19931127 0 840 1 199312 kwbc fcmean 2t 1 19931201 0 19931127 6 840 1 199312 kwbc fcmean 2t 2 19931201 0 19931127 12 840 1 199312 kwbc fcmean 2t 3 19931201 0 19931127 18 840 1 199312 kwbc fcmean 2t 4 19931201 0 19931122 0 960 1 199312 kwbc fcmean 2t 5 19931201 0 19931122 6 960 1 199312 kwbc fcmean 2t 6 19931201 0 19931122 12 960 1 199312 kwbc fcmean 2t 7 19931201 0 19931122 18 960 1 199312 kwbc fcmean 2t 8 19931201 0 19931117 0 1080 1 199312 kwbc fcmean 2t 9 19931201 0 19931117 6 1080 1 199312 kwbc fcmean 2t 10 19931201 0 19931117 12 1080 1 199312 kwbc fcmean 2t 11 19931201 0 19931117 18 1080 1 199312 kwbc fcmean 2t 12 19931201 0 19931112 0 1200 1 199312 kwbc fcmean 2t 13 19931201 0 19931112 6 1200 1 199312 kwbc fcmean 2t 14 19931201 0 19931112 12 1200 1 199312 kwbc fcmean 2t 15 19931201 0 19931112 18 1200 1 199312 kwbc fcmean 2t 16 19931201 0 19931107 0 1320 1 199312 kwbc fcmean 2t 17 19931201 0 19931107 6 1320 1 199312 kwbc fcmean 2t 18 19931201 0 19931107 12 1320 1 199312 kwbc fcmean 2t 19 19931201 0 19931107 18 1320 1 199312 kwbc fcmean 2t 20 19931201 0 19931102 0 1440 1 199312 kwbc fcmean 2t 21 19931201 0 19931102 6 1440 1 199312 kwbc fcmean 2t 22 19931201 0 19931102 12 1440 1 199312 kwbc fcmean 2t 23 19931201 0 19931102 18 1440 1 199312 kwbc fcmean 2t 0 19931201 0 19931127 0 1584 2 199401 kwbc fcmean 2t 1 19931201 0 19931127 6 1584 2 199401 kwbc fcmean 2t 2 19931201 0 19931127 12 1584 2 199401 kwbc fcmean 2t 3 19931201 0 19931127 18 1584 2 199401 kwbc fcmean 2t 4 19931201 0 19931122 0 1704 2 199401 kwbc fcmean 2t 5 19931201 0 19931122 6 1704 2 199401 kwbc fcmean 2t 6 19931201 0 19931122 12 1704 2 199401 kwbc fcmean 2t 7 19931201 0 19931122 18 1704 2 199401 kwbc fcmean 2t 8 19931201 0 19931117 0 1824 2 199401 kwbc fcmean 2t 9 19931201 0 19931117 6 1824 2 199401 kwbc fcmean 2t 10 19931201 0 19931117 12 1824 2 199401 kwbc fcmean 2t 11 19931201 0 19931117 18 1824 2 199401 kwbc fcmean 2t 12 19931201 0 19931112 0 1944 2 199401 kwbc fcmean 2t 13 19931201 0 19931112 6 1944 2 199401 kwbc fcmean 2t 14 19931201 0 19931112 12 1944 2 199401 kwbc fcmean 2t 15 19931201 0 19931112 18 1944 2 199401 kwbc fcmean 2t 16 19931201 0 19931107 0 2064 2 199401 kwbc fcmean 2t 17 19931201 0 19931107 6 2064 2 199401 kwbc fcmean 2t 18 19931201 0 19931107 12 2064 2 199401 kwbc fcmean 2t 19 19931201 0 19931107 18 2064 2 199401 kwbc fcmean 2t 20 19931201 0 19931102 0 2184 2 199401 kwbc fcmean 2t 21 19931201 0 19931102 6 2184 2 199401 kwbc fcmean 2t 22 19931201 0 19931102 12 2184 2 199401 kwbc fcmean 2t 23 19931201 0 19931102 18 2184 2 199401
This document has been produced in the context of the Copernicus Climate Change Service (C3S).
The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of C3S on behalf of the European Union (Delegation agreement signed on 11/11/2014). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose.
The users thereof use the information at their sole risk and liability. For the avoidance of all doubt, the European Commission and the European Centre for Medium-Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view.
