Extended Range - CDFs and EFIs
To complement the EFI and SOT for two parameters in the extended range (introduced with IFS cycle 46r1 in June 2019) a facility to view extended-range Cumulative Distribution Functions (CDFs) for those parameters was also introduced (in February 2020). Unlike ECMWF's pre-existing CDF-viewing tools, used for 24h periods at shorter ranges, which show absolute values, these CDFs depict anomalies (relative to the ER-M-Climate distribution). They cover the following:
2-metre weekly mean temperature anomaly
7-day total precipitation anomaly.
Forecast time steps:
000-168h, 096-264h, 168-336h, 264-432h, 336-504h, 432-600h, 504-672h, 600-768h, 672-840h, 768-936h, 840-1008h, 936-1104h
Fig126.96.36.199: To view Extended Range CDFs (for e.g. Precipitation):
- In the main charts page, click on ecCharts.
- On the ecChart, select the "Layers" tab.
- On the Layers drop down, select (via "Add Layers") the "Total precipitation shift of tails (SOT) index" and "Extended range: EFI for total precipitation".
- On the ecChart header, select "Views".
- Via the "Views" drop down, select any required windows for display.
- On the "Views" drop down, select the Meteogram window.
- If necessary choose the desired CDF option(s), for Extended ranges, using the "More ..." button
- Display of product
Each CDF plot displays the latest forecast along with the Extended Range Model climate (ERM-climate) corresponding to it, and also all previous extended range ensemble forecasts valid for the same 7-day period. The following percentiles are used to draw both ERM-climate and ensemble forecast CDFs: 0 (minimum), 1, 2, 5, 10, 25, 50 (median), 75, 90, 95, 98, 99 and 100 (maximum). For the forecasts, which have far fewer realisations than the ERM-Climate, percentiles 0, 1 and 2 all take the same value, as do 98, 99 and 100.
Fig188.8.131.52: The EFI and SOT for 2-metre weekly mean temperature anomalies, and CDF plots of temperature and precipitation anomalies for one site (see green pin). On the CDF widget the black curve represents the ERM-climate and coloured curves represent the different extended-range forecasts valid for the same 7-day period.
Consideration when interpreting the charts
At these longer forecast times in the extended range the latest run may not necessarily be the best and users may wish to consider more than one set of solutions as a lagged ensemble. In the example shown in Fig184.108.40.206 the latest forecast temperature anomaly CDF (in red) shows a strong positive anomaly while previous forecasts suggested a more modest warm signal.
For both precipitation and temperature, zero on the x-axis (and the thicker vertical gridline) simply corresponds to ERM-climate mean values (for the location, the time of year and the lead time displayed), because of course "anomaly" computations use these mean values as their reference points. This statement is true for all of the curves. However, for different lead times (i.e. the different coloured curves) the absolute value that is the mean will vary a little bit (due to model drift and under-sampling). In spite of such variations it is still reasonable, helpful and recommended to inter-compare the single black ERM-climate curve with all the coloured curves (even if this is only strictly valid for the same lead time that it represents - i.e. the red curve).
Note, incidentally, that the ER-M-climate, as used here and on extended range meteograms, is now based on re-forecasts initialised from ERA5 data; this is higher quality output, and has greater compatibility with actual forecasts, than was the case previously when the re-forecasts were initialised from ERA-Interim data (i.e. before model cycle 46r1 was introduced in June 2019).
There are many different options for how to construct CDF plots - i.e. axis limits etc. All have advantages and disadvantages. The structure ECMWF uses, described below, arose following several iterations, and arguably provides an optimal compromise.
For precipitation, the plot design is such that the x-axis always starts from the ERM-climate minimum (black), for the given valid dates, for the lead time of the latest forecast (red). So the black curve should always begin from the graph's lower left corner (Fig. 220.127.116.11a). Note that over the vast majority of the world this will equate to zero precipitation in the 7-day period; it is of course impossible to get less than zero. As the ERM-climate is a function of lead time the forecasts shown for different lead times (not red), which are referenced to (slightly) different ERM-climates to calculate anomalies, may not have the same "zero" starting point. On Fig. 18.104.22.168a for example the end of the dashed purple curve, corresponding to the T+504-672h forecast, probably also corresponds to zero rain, but lies left of the plot area shown (and so is not visible). Equally, such a curve could end just to the right of the lower left point of the graph, but could still correspond to zero rain in absolute terms. It all depends on how the ERM-climate for the given lead time compares with the ERM-Climate shown (black curve). In some very wet locations the ERM-climate CDF may have never been 0 (i.e. no completely dry weeks). In this case the x-axis would start from a value that in the ERM-Climate (black) does not correspond to zero rain in absolute terms as shown on Fig. 22.214.171.124b.
Fig126.96.36.199: CDF plots for 7-day total precipitation anomalies. The plot design caters for two scenarios: (a) - which is very common - used when the ERM-Climate minimum (black) = 0mm in absolute terms (bottom left corner = 0mm), and (b) - which is very rare - used when the ERM-Climate minimum is >0mm. On (a) the purple curve, for example, does not start from "0" (the point where x- and y-axis intersect) because of lead time dependence of the M-climate. A similar situation in which a curve "disappears" off the left of the plot could occur in case (b), but has not done so on this example.
For temperature, the x-axis starts from the overall minimum encountered within all the displayed CDFs (ERM-climate and ensemble forecasts).
Extended range charts for EFI and SOT are available on ecCharts. These are:
- EFI charts for 2m temperature and total precipitation (see Fig188.8.131.52).
- SOT charts for 2m temperature (two option - quantile 10, for cold, and quantile 90, for warm) and total precipitation.
Fig184.108.40.206 ecChart of Extended Range EFI for weekly T2m (centre) and EFI for weekly precipitation (right) covering the same area. Meteograms and time series of EFI for precipitation and 2m temperature, for the Greenland location shown by the pin, are shown in the panels (left). VT:week ending 5 Aug 2019, DT:00UTC 25 Jul 2019.
Fig220.127.116.11: ecCharts of Extended Range EFI and SOT (quantile 10) for weekly T2m anomaly, from DT:00UTC 25 Jul 2019, for: Fig18.104.22.168A: VT:week ending 5 Aug 2019 (left), Fig22.214.171.124B: VT:week ending 29 Aug 2019 (right). Purple area on map shows where EFI is below -0.9. SOT values (quantile 10) are shown in green boxes, the feint line is where SOT=1. Actual EFI and SOT values at the green pin site are shown in the lowest white box. The extended range meteogram with ERM-climate (red), and the time-series (blue) of both EFI and SOT (quantile 10) values for Nizhniy Novgorod illustrate the expected evolution.
- Fig126.96.36.199A (week ending 5 Aug 2019) highlights an area where 2m temperatures are expected to be very much on the cold side of the ERM-Climate distribution (as taken from re-forecasts). The CDF diagram at Fig 188.8.131.52D would be similar to that for Nizhniy Novgorod.
- Fig184.108.40.206B (week ending 29 Aug 2019) shows that the distribution of possible mean 2m temperatures is overall close to the ERM-Climate distribution, implying that the actual model forecast on this occasion is unable to add much to a (model-free) forecast that purely reflects climatological probabilities. The CDF diagram at Fig 220.127.116.11C would be similar to that for Nizhniy Novgorod.
Note that whilst negative SOT values can be computed, and are displayed in meteogram format (as above), and are illustrated below, we advise the user to generally focus on SOT values that are ≥0.8.
Extended Range CDFs.
Cumulative Distribution Function (CDF)s for ensemble temperature and rainfall forecasts may be constructed from ensemble extended range forecasts. It is important to note that here it is anomalies from the "norm" that are considered rather than absolute temperature or rainfall values. The anomalies for the extended range climate (ER-M-climate) (black line) are the frequencies of departures from the mean (here defined as the "norm") of the ER-M-climate for the date in question (i.e.the light green lines on the diagram indicate the value at 50% probability and marked as 0°C anomaly). Some anomalies are positive, some in the tails of the plot are extremely positive; some are negative, some some in the tails of the plot extremely negative. The CDF for the ENS values is constructed from the anomaly of the temperature forecast by each ensemble member (red line) as a departure from the mean or "norm" of the ER-M-climate.
Extreme Forecast Index (EFI) and Shift of Tails (SOT) are derived in the same way as for the medium range products.
Examples of CDF and derivation of SOT
Example of Temperature CDFs: Upper tail positive SOT
Fig18.104.22.168A: The large positive EFI shows the ENS temperature anomaly distribution is for warmer anomalies much above the ER-M-climate anomaly distribution. The positive upper tail SOT (quantile 90) indicates there are several ENS members predicting extreme warm temperature anomalies (above the 99th ER-M-climate percentile shown by the dashed green line). This suggests a warm temperature anomaly may be confidently forecast (large positive EFI), probably exceptional but not necessarily extreme (SOT +0.4). Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.
Example of Temperature CDFs: Upper tail positive SOT
Fig22.214.171.124B: The large positive EFI shows the ENS temperature anomaly distribution is for warmer anomalies much above the ER-M-climate anomaly distribution. The negative upper tail SOT (quantile 90) indicates generally ENS members are not predicting an extreme temperature anomaly (above the 99th ER-M-climate percentile shown by the dashed green line) - however, note one ENS member (extreme top end of red curve) is predicting an extreme temperature anomaly (above the 99th ER-M-climate percentile) though less extreme than the extreme of ER-M-climate anomaly. This suggests a warm temperature anomaly is confidently forecast (fairly large positive EFI), but will be unexceptional (SOT –0.7) compared with ER-M-climate - but nevertheless one member does suggest a near exceptional warm anomaly is possible.
Example of Temperature CDFs: Lower tail negative SOT
Fig126.96.36.199C: The small positive EFI suggests the frequencies of ENS temperature anomaly distribution is near or a little above the ER-M-climate anomaly distribution. The negative lower tail SOT (quantile 10) indicates generally ENS members are not predicting an extreme temperature anomaly (below the 1st ER-M-climate percentile shown by the dashed green line) - however note one ENS member (extreme bottom end of red curve) is predicting an extreme temperature anomaly (below the 1st M-climate percentile) though less extreme than the ER-M-climate anomaly. This suggests the temperature anomalies are similar to the ER-M-climate anomaly distribution (small positive/negative EFI), and a cold anomaly, if it occurs, will be unexceptional compared with ER-M-climate (SOT –1.7) - but nevertheless one member does suggest an exceptional cold anomaly is possible.
A similar CDF diagram would be obtained at Nizhniy Novgorod in Fig188.8.131.52B above.
Example of Temperature CDFs: Lower tail negative SOT
Fig184.108.40.206D: The large negative EFI shows the frequencies of ENS temperature anomaly distribution is for colder anomalies well below the ER-M-climate anomaly distribution. The positive lower tail SOT (quantile 10) indicates there are several ENS members predicting extreme cold temperature anomalies (below the 1st ER-M-climate percentile shown by the dashed green line). This suggests a cold temperature anomaly may be confidently forecast (large negative EFI), exceptional and probably extreme (SOT +0.95). Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.
A similar CDF diagram would be obtained at Nizhniy Novgorod in Fig220.127.116.11A above.
Examples of Rainfall CDFs
It should be remembered that only upper tail SOT may be derived from rainfall CDFs as clearly there are no rainfall totals below 0mm and no anomaly in the extended range model climatology can exist below this value.
Example of Rainfall CDF: positive SOT
Fig18.104.22.168A: An example CDF for snowfall - snowfall is just considered as equivalent rainfall. Moderately large positive EFI shows the equivalent rainfall anomaly distribution is above the ER-M-climate anomaly distribution. The positive upper tail SOT (quantile 90) indicates there are several ENS members predicting extreme equivalent rainfall anomalies (above the 99th ER-M-climate percentile shown by the dashed green line). This suggests uncertainty that a significant equivalent rainfall anomaly is forecast (moderate EFI. Note: 50% of ENS members forecast less than about 1mm precipitation (the lower part of the ER-M-climate only just above 0mm), but equally 25% of ENS members forecast more than about 2mm precipitation (significantly above ER-M-climate where about 97% of precipitation less than 2mm). If a significant rainfall occurs it could be an exceptional rainfall equivalent (SOT 0.8). Confidence in extreme rainfall rises as SOT values increase - users should focus on SOT values >0.5.
Example of Rainfall CDF: negative SOT
Fig22.214.171.124B: The moderate positive EFI suggests the ENS rainfall anomaly distribution is slightly above the ERM-climate anomaly distribution. The negative upper tail SOT (quantile 90) indicates there are very few if any ENS members predicting extreme equivalent rainfall anomalies (above the 99th ER-M-climate percentile shown by the dashed green line). This suggests uncertainty that larger than normal rainfall anomaly may be forecast (moderate EFI - but note 70% of ENS members forecast less than about 1mm precipitation, equally 15% of ENS members forecast more than about 2mm precipitation), but it is unlikely there will be an exceptional rainfall event (SOT -0.6).
Example of Rainfall CDF: negative SOT
Fig126.96.36.199C: An example of a rainfall CDF most frequently encountered where very few ENS members forecast any rain at all. The small negative EFI shows the ENS rainfall anomaly distribution is lower than the ER-M-climate anomaly distribution. The negative upper tail SOT (quantile 90) indicates there are very few if any ENS members (and in this case none of them) predict extreme equivalent rainfall anomalies (above the 99th ER-M-climate percentile shown by the dashed green line). This suggests confidence that larger than normal rainfall anomaly will not occur (small EFI) and an exceptional rainfall event will not occur(SOT -1.2).
Reliability diagrams in Extended Range.
Reliability diagrams are available for extended range forecasts. This gives an assessment of the current model characteristics and allows some indication of the confidence one can have in the evolution shown within the extended ranges - unless there is good evidence to the contrary (e.g. a major change from previous forecasts in the evolution within the medium range).
Fig188.8.131.52: Illustrative examples of Reliability and ROC diagrams (here at week 1 and week 5) highlighting differences in model performance as forecasts progress.
Fig184.108.40.206: Example of Cost/Loss diagrams (here at week1 and week5) illustrating two things in particular: that potential economic value for all users reduces as lead time increases, and that the range of users for whom the forecasts can have some intrinsic economic value reduces markedly as lead time increases.
See Section 12B for description of the Reliability diagrams and ROC diagrams and interpretation of Cost/Loss diagrams.