Section 8.2.4 Extended Range - CDFs, EFI and SOT

Extended Range - CDFs and EFIs

ecChart display.

Extended range extreme forecast index (EFI), shift of tails (SOT) and cumulative distribution functions (CDF) is available for 2m temperature and precipitation.  These CDFs depict anomalies (relative to the ER-M-Climate distribution).  This is unlike ECMWF the CDF used for 24h periods at shorter ranges which show absolute values.  The CDFs cover the following:

Parameters:

7-day mean 2m temperature anomaly
7-day total precipitation anomaly.

Forecast time steps: 

000-168h, 096-264h, 168-336h, 264-432h, 336-504h, 432-600h, 504-672h, 600-768h, 672-840h, 768-936h, 840-1008h, 936-1104h 

Fig8.2.4-1: To view Extended Range CDFs (for e.g. Precipitation):

1. In the main charts page, click on ecCharts.
2. On the ecChart, select the "Layers" tab.
3. On the Layers drop down, select (via "Add Layers") the "Total precipitation shift of tails (SOT) index" and "Extended range: EFI for total precipitation".
4. On the ecChart header, select "Views".
5. Via the "Views" drop down, select any required windows for display. 
6. On the "Views" drop down, select the Meteogram window.
7. If necessary choose the desired CDF option(s), for Extended ranges, using the "More ..." button
8. Display of product 

Each CDF plot displays the latest forecast along with the Extended Range Model climate (ER-M-climate) corresponding to it, and also all previous extended range ensemble forecasts valid for the same 7-day period. The following percentiles are used to draw both ER-M-climate and ensemble forecast CDFs: 0 (minimum), 1, 2, 5, 10, 25, 50 (median), 75, 90, 95, 98, 99 and 100 (maximum). For the forecasts, which have far fewer realisations than the ER-M-climate, percentiles 0, 1 and 2 all take the same value, as do 98, 99 and 100.

Fig8.2.4-2: The EFI and SOT for 2-metre weekly mean temperature anomalies, and CDF plots of temperature and precipitation anomalies for one site (see green pin). On the CDF widget the black curve represents the ER-M-climate and coloured curves represent the different extended-range forecasts valid for the same 7-day period.

Consideration when interpreting the charts

At these longer forecast times in the extended range the latest run may not necessarily be the best and users may wish to consider more than one set of solutions as a lagged ensemble.  In the example shown in Fig8.2.4-2 the latest forecast temperature anomaly CDF (in red) shows a strong positive anomaly while previous forecasts suggested a more modest warm signal.

Plot design.

For both precipitation and temperature, zero on the x-axis (and the thicker vertical gridline) simply corresponds to ER-M-climate mean values (for the location, the time of year and the lead time displayed), because of course "anomaly" computations use these mean values as their reference points. This statement is true for all of the curves. However, for different lead times (i.e. the different coloured curves) the absolute value that is the mean will vary a little bit (due to model drift and under-sampling). In spite of such variations it is still reasonable, helpful and recommended to inter-compare the single black ER-M-climate curve with all the coloured curves (even if this is only strictly valid for the same lead time that it represents - i.e. the red curve).

Note, incidentally, that the ER-M-climate, as used here and on extended range meteograms, is now based on re-forecasts initialised from ERA5 data; this is higher quality output, and has greater compatibility with actual forecasts, than was the case previously when the re-forecasts were initialised from ERA-Interim data (i.e. before model cycle 46r1 was introduced in June 2019).

There are many different options for how to construct CDF plots - i.e. axis limits etc. All have advantages and disadvantages. The structure ECMWF uses, described below, arose following several iterations, and arguably provides an optimal compromise.

For precipitation, the plot design is such that the x-axis always starts from the ER-M-climate minimum (black), for the given valid dates, for the lead time of the latest forecast (red). So the black curve should always begin from the graph's lower left corner (Fig8.2.4-3). Note that over the vast majority of the world this will equate to zero precipitation in the 7-day period; it is of course impossible to get less than zero. As the ER-M-climate is a function of lead time the forecasts shown for different lead times (not red), which are referenced to (slightly) different ER-M-climates to calculate anomalies, may not have the same "zero" starting point. On Fig8.2.4-3 for example the end of the dashed purple curve, corresponding to the T+504-672h forecast, probably also corresponds to zero rain, but lies left of the plot area shown (and so is not visible). Equally, such a curve could end just to the right of the lower left point of the graph, but could still correspond to zero rain in absolute terms. It all depends on how the ER-M-climate for the given lead time compares with the ER-M-climate shown (black curve). In some very wet locations the ER-M-climate CDF may have never been 0 (i.e. no completely dry weeks). In this case the x-axis would start from a value that in the ER-M-climate (black) does not correspond to zero rain in absolute terms as shown on Fig8.2.4-3.

a)b)

Fig8.2.4-3: CDF plots for 7-day total precipitation anomalies. The plot design caters for two scenarios: (a) - which is very common - used when the ER-M-climate minimum (black) = 0mm in absolute terms (bottom left corner = 0mm), and (b) - which is very rare - used when the ER-M-climate minimum is >0mm. On (a) the purple curve, for example, does not start from "0" (the point where x- and y-axis intersect) because of lead time dependence of the M-climate. A similar situation in which a curve "disappears" off the left of the plot could occur in case (b), but has not done so on this example.

For temperature, the x-axis starts from the overall minimum encountered within all the displayed CDFs (ER-M-climate and ensemble forecasts).

An example.

Extended range charts for EFI and SOT are available on ecCharts.  These are:

• EFI charts for 2m temperature and total precipitation (see Fig8.2.4-4).
• SOT charts for 2m temperature (two option - quantile 10, for cold, and quantile 90, for warm) and total precipitation.

Fig8.2.4-4 ecChart of Extended Range EFI for weekly T2m (centre) and EFI for weekly precipitation (right) covering the same area.  Meteograms and time series of EFI for precipitation and 2m temperature, for the Greenland location shown by the pin, are shown in the panels (left).  VT:week ending 5 Aug 2019, DT:00UTC 25 Jul 2019.

Fig8.2.4-5: ecChart of Extended Range EFI and SOT (quantile 10) for weekly T2m anomaly, from DT:00UTC 25 Jul 2019, VT: week ending 5 Aug 2019.  The chart highlights an area where 2m temperatures are expected to be very much on the cold side of the ER-M-climate distribution (as taken from re-forecasts).  The CDF diagram at Fig8.2.4-3(a) would be similar to that for Nizhniy Novgorod.  The purple area on map shows where EFI is below -0.9.  SOT values (quantile 10) are shown in green boxes, the feint line is where SOT=1. Actual EFI and SOT values at the green pin site are shown in the lowest white box. The extended range meteogram with ER-M-climate (red), and the time-series (blue) of both EFI and SOT (quantile 10) values for Nizhniy Novgorod illustrate the expected evolution.

Fig8.2.4-6:  ecChart of Extended Range EFI and SOT (quantile 10) for weekly T2m anomaly, from DT:00UTC 25 Jul 2019, VT:week ending 29 Aug 2019.  The chart shows that the distribution of possible mean 2m temperatures is overall close to the ER-M-climate distribution, implying that the actual model forecast on this occasion is unable to add much to a (model-free) forecast that purely reflects climatological probabilities.  The CDF diagram at Fig8.2.4-3(b) would be similar to that for Nizhniy Novgorod.

Note that whilst negative SOT values can be computed, and are displayed in meteogram format (as above), and are illustrated below, we advise the user to generally focus on SOT values that are ≥0.8.

OpenChart display

Extreme Forecast Index (EFI) - Extended Range Forecast

These EFI charts aim to point to areas where unusually anomalous temperature or precipitation is likely to occur.

The EFI temperature chart shows the weekly mean EFI for 2 m temperatures.  This is derived from the distribution of ensemble forecast 2 m temperatures compared with the temperature distribution in the ER-M-climate.

The EFI precipitation chart shows the weekly mean EFI for precipitation.  This is derived from the distribution of ensemble forecast precipitation compared with the precipitation distribution in the ER-M-climate.

Click on the central small icon in the bottom right of the web frame to show the colour scale of values appropriate to each display.

Experience suggests :

• EFI values between 0.5 to 0.8 (irrespective of sign) can be generally regarded as signifying that “unusual” weather is likely,
• EFI values greater than 0.8 (irrespective of sign) usually signifies that “very unusual” or extreme weather is likely.

The SOT index provides information about how extreme an event could potentially be.  Positive SOT values indicate that at least 10% of the ensemble is forecasting an "extreme event" and a high value shows how extreme:

• Dashed black isopleths show SOT values associated with the 10% of ENS results (quantile 10) showing the coldest temperatures.
• Solid black isopleths show SOT values associated with the 10% of ENS results (quantile 90) showing the warmest temperatures or most precipitation.

Extended Range CDFs.

Cumulative Distribution Function (CDF)s for ensemble temperature and rainfall forecasts may be constructed from ensemble extended range forecasts.   It is important to note that here it is anomalies from the "norm" that are considered rather than absolute temperature or rainfall values.  The anomalies for the extended range climate (ER-M-climate) (black line) are the frequencies of departures from the mean (here defined as the "norm") of the ER-M-climate for the date in question (i.e.the light green lines on the diagram indicate the value at 50% probability and marked as 0°C anomaly).   Some anomalies are positive, some in the tails of the plot are extremely positive; some are negative, some some in the tails of the plot extremely negative.   The CDF for the ensemble values is constructed from the anomaly of the temperature forecast by each ensemble member (red line) as a departure from the mean or "norm" of the ER-M-climate.

Extreme Forecast Index (EFI) and Shift of Tails (SOT) are derived in the same way as for the medium range products.

Examples of CDF and derivation of SOT

Example of Temperature CDFs: Upper tail positive SOT

Fig8.2.4-7: The large positive EFI shows the ensemble temperature anomaly distribution is for warmer anomalies much above the ER-M-climate anomaly distribution.  The positive upper tail SOT (quantile 90) indicates there are several ensemble members predicting extreme warm temperature anomalies (above the 99th ER-M-climate percentile shown by the dashed green line).  This suggests a warm temperature anomaly may be confidently forecast (large positive EFI), probably exceptional but not necessarily extreme (SOT +0.4).  Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.

Example of Temperature CDFs: Upper tail positive SOT

Fig8.2.4-8: The large positive EFI shows the ensemble temperature anomaly distribution is for warmer anomalies much above the ER-M-climate anomaly distribution.  The negative upper tail SOT (quantile 90) indicates generally ensemble members are not predicting an extreme temperature anomaly (above the 99th ER-M-climate percentile shown by the dashed green line) - however, note one ensemble member (extreme top end of red curve) is predicting an extreme temperature anomaly (above the 99th ER-M-climate percentile) though less extreme than the extreme of ER-M-climate anomaly.   This suggests a warm temperature anomaly is confidently forecast (fairly large positive EFI), but will be unexceptional (SOT –0.7) compared with ER-M-climate - but nevertheless one member does suggest a near exceptional warm anomaly is possible.

Example of Temperature CDFs: Lower tail negative SOT

Fig8.2.4-9: The small positive EFI suggests the frequencies of ensemble temperature anomaly distribution is near or a little above the ER-M-climate anomaly distribution.  The negative lower tail SOT (quantile 10) indicates generally ensemble members are not predicting an extreme temperature anomaly (below the 1st ER-M-climate percentile shown by the dashed green line) - however note one ensemble member (extreme bottom end of red curve) is predicting an extreme temperature anomaly (below the 1st M-climate percentile) though less extreme than the ER-M-climate anomaly.   This suggests the temperature anomalies are similar to the ER-M-climate anomaly distribution (small positive/negative EFI), and a cold anomaly, if it occurs, will  be unexceptional compared with ER-M-climate (SOT –1.7) - but nevertheless one member does suggest an exceptional cold anomaly is possible.

A similar CDF diagram would be obtained at Nizhniy Novgorod in Fig8.2.4-6 above.

Example of Temperature CDFs: Lower tail negative SOT

Fig8.2.4-10: The large negative EFI shows the frequencies of ensemble temperature anomaly distribution is for colder anomalies well below the ER-M-climate anomaly distribution.  The positive lower tail SOT (quantile 10)  indicates there are several ensemble members predicting extreme cold temperature anomalies (below the 1st ER-M-climate percentile shown by the dashed green line).   This suggests a cold temperature anomaly may be confidently forecast (large negative EFI), exceptional and probably extreme (SOT +0.95).  Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.

A similar CDF diagram would be obtained at Nizhniy Novgorod in Fig8.2.4-5 above.

Examples of Rainfall CDFs

It should be remembered that only upper tail SOT may be derived from rainfall CDFs as clearly there are no rainfall totals below 0mm and no anomaly in the extended range model climatology can exist below this value.

Example of Rainfall CDF: positive SOT 

Fig8.2.4-11: An example CDF for snowfall - snowfall is just considered as equivalent rainfall.   Moderately large positive EFI shows the equivalent rainfall anomaly distribution is above the ER-M-climate anomaly distribution.  The positive upper tail SOT (quantile 90) indicates there are several ensemble members predicting extreme equivalent rainfall anomalies (above the 99th ER-M-climate percentile shown by the dashed green line).  This suggests uncertainty that a significant equivalent rainfall anomaly is forecast (moderate EFI.  Note: 50% of ensemble members forecast less than about 1mm precipitation (the lower part of the ER-M-climate only just above 0mm), but equally 25% of ensemble members forecast more than about 2mm precipitation (significantly above ER-M-climate where about 97% of precipitation less than 2mm). If a significant rainfall occurs it could be an exceptional rainfall equivalent (SOT 0.8).  Confidence in extreme rainfall rises as SOT values increase - users should focus on SOT values >0.5.

Example of Rainfall CDF: negative SOT  

Fig8.2.4-12: The moderate positive EFI suggests the ensemble rainfall anomaly distribution is slightly above the ER-M-climate anomaly distribution.  The negative upper tail SOT (quantile 90) indicates there are very few if any ensemble members predicting extreme equivalent rainfall anomalies (above the 99th ER-M-climate percentile shown by the dashed green line).  This suggests uncertainty that larger than normal rainfall anomaly may be forecast (moderate EFI - but note 70% of ensemble members forecast less than about 1mm precipitation, equally 15% of ensemble members forecast more than about 2mm precipitation), but it is unlikely there will be an exceptional rainfall event (SOT -0.6).

Example of Rainfall CDF: negative SOT 

Fig8.2.4-13: An example of a rainfall CDF most frequently encountered where very few ensemble members forecast any rain at all.  The small negative EFI shows the ensemble rainfall anomaly distribution is lower than the ER-M-climate anomaly distribution.  The negative upper tail SOT (quantile 90) indicates there are very few if any ensemble members (and in this case none of them) predict extreme equivalent rainfall anomalies (above the 99th ER-M-climate percentile shown by the dashed green line).  This suggests confidence that larger than normal rainfall anomaly will not occur (small EFI) and an exceptional rainfall event will not occur(SOT -1.2).

Reliability diagrams in Extended Range.

Reliability diagrams are available for extended range forecasts. This gives an assessment of the current model characteristics and allows some indication of the confidence one can have in the evolution shown within the extended ranges - unless there is good evidence to the contrary (e.g. a major change from previous forecasts in the evolution within the medium range).

Fig8.2.4-14: Illustrative examples of Reliability and ROC diagrams (here at week 1 and week 5) highlighting differences in model performance as forecasts progress.

.  

Fig8.2.4-15: Example of Cost/Loss diagrams (here at week1 and week5) illustrating two things in particular: that potential economic value for all users reduces as lead time increases, and that the range of users for whom the forecasts can have some intrinsic economic value reduces markedly as lead time increases.

See Section 12B for description of the Reliability diagrams and ROC diagrams and interpretation of Cost/Loss diagrams.