1. Context.The concept of "nominal" start date in lagged start ensembles
Within the seasonal forecast datasets available at C3S, we have contributions from two families of forecasting systems, depending on what techniques are used to build the members of their ensemble forecast, so that they sample the uncertainty in the initial conditions:
- "Burst" mode: all the members are initialized with conditions on the same start date, but from slightly different (perturbed) initial states, intended to sample the uncertainty in observations. (e.g. all members initialized on 1st March 2017; this is the case for ECMWF's system)
- "Lagged" mode: members are initialized on different start dates, the differences between which are sufficiently small (e.g. members initialized every day of the month; this is the case for Met Office system)
Due to a number of reasons, members from ensembles using the "lagged start" approach are bundled together at C3S and labelled with a "nominal" start date (see details here).
The C3S seasonal forecast datasets are based on GRIB files stored at ECMWF MARS archive where that "nominal" start date is used to index the monthly data (monthly statistics and monthly post-processed products like anomalies). Hence, the details about the original initialization date could get lost in those files. Nonetheless, the data at daily and sub-daily time frequencies is always encoded with the real initialization date, as it can be seen in the example below:
Daily/Sub-daily data | Monthly data | ||
Date | Ensemble member | Date | Ensemble member |
19931127 | 0 | 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 19931201 | 0 |
2. The encoding in GRIB format using "Local Definitions"
In December 2019, with the release of ecCodes 2.15.0, an extension to the GRIB encoding was introduced to allow storing additionally the original initialization date on those monthly files, while keeping the "nominal" start date as indexing date to be used by the MARS archive. The solution adopted was to create a new GRIB "local definition" for the seasonal forecast monthly data produced by lagged start systems.
Up to that date, all seasonal forecast monthly data files produced at ECMWF, and that includes the data provided by all C3S seasonal forecast contributors, was encoded using GRIB local definitions described in table 16 (GRIB keyword localDefinitionNumber=16
). With the changes introduced with version 2.15.0 of ecCodes a new local definition has been devised for lagged start systems (GRIB keyword localDefinitionNumber=12
) and its use has been implemented to all the lagged start systems data produced after December 2019.
Sytems affected by this change
- MetOffice GloSea5 (system=14) used
localDefinitionNumber=16
up to December 2019 nominal start date, and started usinglocalDefinitionNumber=12
from January 2020 onwards. (Note this affects not just the forecasts but also the relevant hindcasts) - NCEP CFSv2 uses
localDefinitionNumber=12
since its introduction into the C3S multi-system (Note that this was achieved by delaying the publication of all monthly data until December 2019)
In the future, systems using the lagged start approach introduced into C3S will also use localDefinitionNumber=12
What has changed?
| localDefinitionNumber=12 (after December 2019) | |
---|---|---|
Date used in the CDS requests | Nominal start date GRIB keywords: dataDate , dataTime | Nominal start date GRIB keywords: indexingDate , indexingTime |
Real start date | Not available | GRIB keywords: dataDate , dataTime |
It is important to bear in mind that most common grib tools will read dataDate
and dataTime
It is therefore recommended to use tools based on ecCodes
(version 2.15.0 or greater), specifically cfgrib
(version 0.9.7.4 or greater) which can be used within python's xarray
3. Examples
Example 1: UK MetOffice hindcast data for December 1993 (localDefinitionNumber=16
)
import cdsapi c = cdsapi.Client() c.retrieve( 'seasonal-monthly-single-levels', { 'format': 'grib', 'originating_centre': 'ukmo', 'system': '14', 'variable': '2m_temperature', 'product_type': 'monthly_mean', 'year': '1993', 'month': '12', 'leadtime_month': [ '1', '2', ], }, 'c3s_seas_ukmo199312_t2m.grib')
> grib_ls -p origin,type,shortName,number,dataDate,dataTime,indexingDate,indexingTime,step,fcmonth,verifyingMonth c3s_seas_ukmo199312_t2m.grib c3s_seas_ukmo199312_t2m.grib origin type shortName number dataDate dataTime indexingDate indexingTime step fcmonth verifyingMonth egrr fcmean 2t 0 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 1 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 2 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 3 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 4 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 5 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 6 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 7 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 8 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 9 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 10 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 11 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 12 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 13 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 14 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 15 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 16 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 17 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 18 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 19 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 20 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 21 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 22 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 23 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 24 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 25 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 26 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 27 19931201 0 not_found not_found 744 1 199312 egrr fcmean 2t 0 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 1 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 2 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 3 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 4 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 5 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 6 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 7 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 8 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 9 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 10 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 11 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 12 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 13 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 14 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 15 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 16 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 17 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 18 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 19 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 20 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 21 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 22 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 23 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 24 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 25 19931201 0 not_found not_found 1488 2 199401 egrr fcmean 2t 26 19931201 0 not_found not_found 1488 2 199401
Example 2: NCEP hindcast data for December 1993 (localDefinitionNumber=12
)
import cdsapi c = cdsapi.Client() c.retrieve( 'seasonal-monthly-single-levels', { 'format': 'grib', 'originating_centre': 'ncep', 'system': '2', 'variable': '2m_temperature', 'product_type': 'monthly_mean', 'year': '1993', 'month': '12', 'leadtime_month': [ '1', '2', ], }, 'c3s_seas_ukmo199312_t2m.grib')
> grib_ls -p origin,type,shortName,number,dataDate,dataTime,indexingDate,indexingTime,step,fcmonth,verifyingMonth c3s_seas_ncep199312_t2m.grib c3s_seas_ncep199312_t2m.grib origin type shortName number dataDate dataTime indexingDate indexingTime step fcmonth verifyingMonth kwbc fcmean 2t 0 19931127 0 19931201 0 840 1 199312 kwbc fcmean 2t 1 19931127 6 19931201 0 840 1 199312 kwbc fcmean 2t 2 19931127 12 19931201 0 840 1 199312 kwbc fcmean 2t 3 19931127 18 19931201 0 840 1 199312 kwbc fcmean 2t 4 19931122 0 19931201 0 960 1 199312 kwbc fcmean 2t 5 19931122 6 19931201 0 960 1 199312 kwbc fcmean 2t 6 19931122 12 19931201 0 960 1 199312 kwbc fcmean 2t 7 19931122 18 19931201 0 960 1 199312 kwbc fcmean 2t 8 19931117 0 19931201 0 1080 1 199312 kwbc fcmean 2t 9 19931117 6 19931201 0 1080 1 199312 kwbc fcmean 2t 10 19931117 12 19931201 0 1080 1 199312 kwbc fcmean 2t 11 19931117 18 19931201 0 1080 1 199312 kwbc fcmean 2t 12 19931112 0 19931201 0 1200 1 199312 kwbc fcmean 2t 13 19931112 6 19931201 0 1200 1 199312 kwbc fcmean 2t 14 19931112 12 19931201 0 1200 1 199312 kwbc fcmean 2t 15 19931112 18 19931201 0 1200 1 199312 kwbc fcmean 2t 16 19931107 0 19931201 0 1320 1 199312 kwbc fcmean 2t 17 19931107 6 19931201 0 1320 1 199312 kwbc fcmean 2t 18 19931107 12 19931201 0 1320 1 199312 kwbc fcmean 2t 19 19931107 18 19931201 0 1320 1 199312 kwbc fcmean 2t 20 19931102 0 19931201 0 1440 1 199312 kwbc fcmean 2t 21 19931102 6 19931201 0 1440 1 199312 kwbc fcmean 2t 22 19931102 12 19931201 0 1440 1 199312 kwbc fcmean 2t 23 19931102 18 19931201 0 1440 1 199312 kwbc fcmean 2t 0 19931127 0 19931201 0 1584 2 199401 kwbc fcmean 2t 1 19931127 6 19931201 0 1584 2 199401 kwbc fcmean 2t 2 19931127 12 19931201 0 1584 2 199401 kwbc fcmean 2t 3 19931127 18 19931201 0 1584 2 199401 kwbc fcmean 2t 4 19931122 0 19931201 0 1704 2 199401 kwbc fcmean 2t 5 19931122 6 19931201 0 1704 2 199401 kwbc fcmean 2t 6 19931122 12 19931201 0 1704 2 199401 kwbc fcmean 2t 7 19931122 18 19931201 0 1704 2 199401 kwbc fcmean 2t 8 19931117 0 19931201 0 1824 2 199401 kwbc fcmean 2t 9 19931117 6 19931201 0 1824 2 199401 kwbc fcmean 2t 10 19931117 12 19931201 0 1824 2 199401 kwbc fcmean 2t 11 19931117 18 19931201 0 1824 2 199401 kwbc fcmean 2t 12 19931112 0 19931201 0 1944 2 199401 kwbc fcmean 2t 13 19931112 6 19931201 0 1944 2 199401 kwbc fcmean 2t 14 19931112 12 19931201 0 1944 2 199401 kwbc fcmean 2t 15 19931112 18 19931201 0 1944 2 199401 kwbc fcmean 2t 16 19931107 0 19931201 0 2064 2 199401 kwbc fcmean 2t 17 19931107 6 19931201 0 2064 2 199401 kwbc fcmean 2t 18 19931107 12 19931201 0 2064 2 199401 kwbc fcmean 2t 19 19931107 18 19931201 0 2064 2 199401 kwbc fcmean 2t 20 19931102 0 19931201 0 2184 2 199401 kwbc fcmean 2t 21 19931102 6 19931201 0 2184 2 199401 kwbc fcmean 2t 22 19931102 12 19931201 0 2184 2 199401 kwbc fcmean 2t 23 19931102 18 19931201 0 2184 2 199401