Sub-seasonal range - CDFs and EFIs

ecChart display.

Sub-seasonal range extreme forecast index (EFI), shift of tails (SOT) and cumulative distribution functions (CDF) is available for 2m temperature and precipitation.  These CDFs depict anomalies relative to the SUBS-M-Climate distribution.  This is unlike the CDF for 24h periods for shorter ranges that show absolute values.  The CDFs cover the following:

Parameters:

7-day mean 2m temperature anomaly
7-day total precipitation anomaly.

Forecast time steps: 

000-168h, 096-264h, 168-336h, 264-432h, 336-504h, 432-600h, 504-672h, 600-768h, 672-840h, 768-936h, 840-1008h, 936-1104h 


Fig8.2.4-1: To view sub-seasonal Range CDFs (for e.g. Precipitation):

  1. In the main charts page, click on ecCharts.
  2. On the ecChart, select the "Layers" tab.
  3. On the Layers drop down, select (via "Add Layers") the "Total precipitation shift of tails (SOT) index" and "sub-seasonal range: EFI for total precipitation".
  4. On the ecChart header, select "Views".
  5. Via the "Views" drop down, select any required windows for display. 
  6. On the "Views" drop down, select the Meteogram window.
  7. If necessary choose the desired CDF option(s), for sub-seasonal ranges, using the "More ..." button
  8. Display of product 


Sub-seasonal range CDF plots show the latest (red) and previous (coloured) ensemble forecasts valid for the same 7-day period.   The corresponding sub-seasonal range model climate (SUBS-M-Climate) is shown in black.  The percentiles used to draw both SUBS-M-Climate and ensemble forecast CDFs are:   0 (minimum), 1, 2, 5, 10, 25, 50 (median), 75, 90, 95, 98, 99 and 100 (maximum).  When forecasts have far fewer realisations than the sub-seasonal range model climate then:

  • percentiles 0, 1 and 2 are assigned the same value if this occurs in the CDF lower tail. 
  • percentiles 98, 99 and 100 are assigned same value. if this occurs in the CDF upper tail. 


Fig8.2.4-2: The EFI and SOT for 2-metre weekly mean temperature anomalies, and CDF plots of temperature and precipitation anomalies for one site (see green pin). On the CDF widget the black curve represents the SUBS-M-Climate and coloured curves represent the different sub-seasonal range forecasts valid for the same 7-day period.

Consideration when interpreting the charts

At these longer forecast times in the sub-seasonal range the latest run may not necessarily be the best and users may wish to consider more than one set of solutions as a lagged ensemble.  In the example shown in Fig8.2.4-2 the latest forecast temperature anomaly CDF (in red) shows a strong positive anomaly while previous forecasts suggested a more modest warm signal.

Plot design.

For both precipitation and temperature, zero on the x-axis (and the thicker vertical gridline) simply corresponds to SUBS-M-Climate mean values (for the location, the time of year and the lead time displayed), because of course "anomaly" computations use these mean values as their reference points. This statement is true for all of the curves.    However, for different lead times (different coloured curves) the absolute value that is the mean varies a little due to model drift and under-sampling.   Strictly, the SUBS-M-Climate curve (black) is valid for the same lead time of the latest ensemble (red).  Despite this, all curves from earlier ensembles (coloured) should be compared with the SUBS-M-Climate curve (black) as well.    

Note, incidentally, that the SUBS-M-Climate, as used here and on sub-seasonal range meteograms, is now based on re-forecasts initialised from ERA5 data; this is higher quality output, and has greater compatibility with actual forecasts, than was the case previously when the re-forecasts were initialised from ERA-Interim data (i.e. before model cycle 46r1 was introduced in June 2019).

There are many different options for how to construct CDF plots - i.e. axis limits etc. All have advantages and disadvantages. The structure ECMWF uses, described below, arose following several iterations, and arguably provides an optimal compromise.

For precipitation:

The CDF is drawn for the given valid period and the lead time of the latest forecast (red).

The CDF x-axis always starts from the SUBS-M-Climate (black) minimum for the given valid dates, for the lead time of the latest forecast (red) (Fig8.2.4-3).  Over the vast majority of the world this will be 0mm precipitation in the 7-day period; it is of course impossible to get less than 0mm.

The SUBS-M-Climate is a function of lead time.  So the other forecasts shown for different lead times (coloured), are referenced to (slightly) different SUBS-M-Climates to calculate anomalies.  They may not have the same 0mm starting point on the diagram. 

On Fig8.2.4-3, the end of the dashed purple curve, corresponding to the T+504-672h forecast, probably also corresponds to 0mm rain, but lies left of the plot area shown (and so is not visible).   Equally, such a curve could end just to the right of the lower left point of the graph, but could still correspond to zero rain in absolute terms. It all depends on how the SUBS-M-Climate for the given lead time compares with the indicated SUBS-M-Climate (black curve).  In some very wet locations the SUBS-M-Climate CDF may have never been 0mm (i.e. no completely dry weeks).  In this case the x-axis would start from a value that in the SUBS-M-Climate (black) does not correspond to 0mm rain in absolute terms as shown on Fig8.2.4-3.

a)b)

Fig8.2.4-3: CDF plots for 7-day total precipitation anomalies. The plot design caters for two scenarios.

  • (a) - which is very common - used when the SUBS-M-Climate minimum (black) = 0mm (bottom left corner = 0mm)
  • (b) - which is very rare - used when the SUBS-M-Climate minimum is >0mm. 

Note: on (a) the purple curve does not start from 0 because of lead time dependence of the M-climate. A similar situation in which a curve "disappears" off the left of the plot could occur in case (b), but has not done so on this example.

For temperature, the x-axis starts from the overall minimum encountered within all the displayed CDFs (SUBS-M-Climate and ensemble forecasts).

An example.

Sub-seasonal range charts for EFI and SOT are available on ecCharts.  These are:

  • EFI charts for 2m temperature and total precipitation (see Fig8.2.4-4).
  • SOT charts for 2m temperature (two option - quantile 10, for cold, and quantile 90, for warm) and total precipitation.


Fig8.2.4-4 ecChart of sub-seasonal Range EFI for weekly T2m (centre) and EFI for weekly precipitation (right) covering the same area.  Meteograms and time series of EFI for precipitation and 2m temperature, for the Greenland location shown by the pin, are shown in the panels (left).  VT:week ending 5 Aug 2019, DT:00UTC 25 Jul 2019.

 

Fig8.2.4-5: ecChart of sub-seasonal Range EFI and SOT (quantile 10) for weekly T2m anomaly, from DT:00UTC 25 Jul 2019, VT: week ending 5 Aug 2019.  The chart highlights an area where 2m temperatures are expected to be very much on the cold side of the SUBS-M-Climate distribution (as taken from re-forecasts).  The CDF diagram at Fig8.2.4-3(a) would be similar to that for Nizhniy Novgorod.  The purple area on map shows where EFI is below -0.9.  SOT values (quantile 10) are shown in green boxes, the feint line is where SOT=1. Actual EFI and SOT values at the green pin site are shown in the lowest white box. The sub-seasonal range meteogram with SUBS-M-Climate (red), and the time-series (blue) of both EFI and SOT (quantile 10) values for Nizhniy Novgorod illustrate the expected evolution.

Fig8.2.4-6:  ecChart of sub-seasonal Range EFI and SOT (quantile 10) for weekly T2m anomaly, DT:00UTC 25 Jul 2019, VT:week ending 29 Aug 2019.  The chart shows that the distribution of possible mean 2m temperatures is overall close to the SUBS-M-Climate distribution.  On this occasion, the ensemble forecast is unable to add much to a forecast that purely reflects climatological probabilities.  At Nizhniy Novgorod the trace from the most recent ensemble forecast would lie close to the climatology trace.


The user should generally focus on SOT values that are ≥0.8.

OpenChart display

Extreme Forecast Index (EFI) - sub-seasonal range forecast

These EFI charts aim to point to areas where unusually anomalous temperature or precipitation is likely to occur.

The EFI temperature chart shows the weekly mean EFI for 2 m temperatures.  This is derived from the distribution of ensemble forecast 2 m temperatures compared with the temperature distribution in the SUBS-M-Climate.

The EFI precipitation chart shows the weekly mean EFI for precipitation.  This is derived from the distribution of ensemble forecast precipitation compared with the precipitation distribution in the SUBS-M-Climate.

Click on the central small icon in the bottom right of the web frame to show the colour scale of values appropriate to each display.

Experience suggests:

  • EFI values between 0.5 to 0.8 (irrespective of sign) can be generally regarded as signifying that “unusual” weather is likely,
  • EFI values greater than 0.8 (irrespective of sign) usually signifies that “very unusual” or extreme weather is likely.

The SOT index provides information about how extreme an event could potentially be.  Positive SOT values indicate that at least 10% of the ensemble is forecasting an "extreme event" and a high value shows how extreme:

  • Dashed black isopleths show SOT values associated with the 10% of ENS results (quantile 10) showing the coldest temperatures.
  • Solid black isopleths show SOT values associated with the 10% of ENS results (quantile 90) showing the warmest temperatures or most precipitation.


Sub-seasonal range CDFs.

Cumulative Distribution Function (CDF)s for ensemble temperature and rainfall forecasts may be constructed from ensemble sub-seasonal range forecasts.   It is important to note that here it is anomalies from the "norm" that are considered rather than absolute temperature or rainfall values.  The anomalies for the sub-seasonal range climate (SUBS-M-Climate) (black line) are the frequencies of departures from the mean (here defined as the "norm") of the SUBS-M-Climate for the date in question (i.e. the light green lines on the diagram indicate the value at 50% probability and marked as 0°C anomaly).   Some anomalies are positive, some in the tails of the plot are extremely positive; some are negative, some some in the tails of the plot extremely negative.   The CDF for the ensemble values is constructed from the anomaly of the temperature forecast by each ensemble member (red line) as a departure from the mean or "norm" of the SUBS-M-Climate.

Extreme Forecast Index (EFI) and Shift of Tails (SOT) are derived in the same way as for the medium range products.

Examples of CDF and derivation of SOT

Example of Temperature CDFs: Upper tail positive SOT

Fig8.2.4-7:  CDF of the ensemble forecasts (red curve) lies to the right of the  SUBS-M-Climate anomaly distribution (black curve).  The ensemble temperature anomaly distribution shows warm anomalies.  The area between the lines shows a large positive EFI.  The positive upper tail SOT (quantile 90) indicates there are several ensemble members with an extreme temperature anomaly above the 99th SUBS-M-Climate percentile (dashed green line).  

The diagram implies a confident forecast of a warm (large positive EFI), probably exceptional (SOT positive ~ +0.4)) but not necessarily extreme temperature anomaly, compared with SUBS-M-Climate.  Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.



Example of Temperature CDFs: Upper tail positive SOT

Fig8.2.4-8: CDF of the ensemble forecasts (red curve) lies to the right of the  SUBS-M-Climate anomaly distribution (black curve).  The ensemble temperature anomaly distribution shows warm anomalies.  The area between the lines shows a large positive EFI.  The negative upper tail SOT (quantile 90) indicates generally ensemble members are not predicting an extreme temperature anomaly above the 99th SUBS-M-Climate percentile (dashed green line).  Note, one ensemble member (extreme top end of red curve) predicts an extreme temperature anomaly (above the 99th SUBS-M-Climate percentile).  However, it is less extreme than the extreme of SUBS-M-Climate anomaly (extreme top end of black curve).   

The diagram implies a confident forecast of a warm (fairly large positive EFI), but unexceptional (SOT negative ~ -0.7) temperature anomaly, compared with SUBS-M-Climate.  However, one member suggests a possible near exceptional warm anomaly.

Example of Temperature CDFs: Lower tail negative SOT

Fig8.2.4-9: CDF of the ensemble forecasts (red curve) lies partly to the right and partly to the left of the  SUBS-M-Climate anomaly distribution (black curve).  The ensemble temperature anomaly distribution shows varying anomalies.  The area between the lines shows only a small EFI.  The negative lower tail SOT (quantile 10) indicates generally ensemble members are not predicting an extreme temperature anomaly below the 1st SUBS-M-Climate percentile (dashed green line).  Note, one ensemble member (extreme lower end of red curve) predicts an extreme temperature anomaly (below the 1st SUBS-M-Climate percentile) .  However, it is less extreme than the extreme of SUBS-M-Climate anomaly (extreme lower end of black curve).   

The diagram implies the temperature anomalies are similar to the SUBS-M-Climate (small EFI), but unexceptional (SOT negative ~ -1.7) temperature anomaly, compared with SUBS-M-Climate.  However, one member suggests a possible near exceptional cold anomaly.

A similar CDF diagram would be obtained at Nizhniy Novgorod in Fig8.2.4-6 above.

Example of Temperature CDFs: Lower tail negative SOT

Fig8.2.4-10:  CDF of the ensemble forecasts (red curve) lies to the left of the  SUBS-M-Climate anomaly distribution (black curve).  The ensemble temperature anomaly distribution shows cold anomalies.  The area between the lines shows a large negative EFI.  The positive lower tail SOT (quantile 10) indicates there are several ensemble members with an extreme temperature anomaly below the 1st SUBS-M-Climate percentile (dashed green line).  

The diagram implies a confident forecast of a cold (large negative EFI), exceptional and probably extreme (SOT positive ~ +0.95), compared with SUBS-M-Climate.  Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.

A similar CDF diagram would be obtained at Nizhniy Novgorod in Fig8.2.4-5 above.

Examples of Rainfall CDFs

It should be remembered that only upper tail SOT may be derived from rainfall CDFs.  Because there are no rainfall totals below 0mm no anomaly in the sub-seasonal range model climatology can exist below this value.

Example of Rainfall CDF: positive SOT 

Fig8.2.4-11: An example CDF for snowfall - snowfall is just considered as equivalent rainfall.

CDF of the ensemble anomaly forecasts (red curve) lies to the right of the  SUBS-M-Climate anomaly distribution (black curve).  The ensemble rainfall anomaly distribution shows anomalies greater than SUBS-M-Climate anomaly distribution.  The area between the lines shows a large positive EFI.  The positive upper tail SOT (quantile 90) indicates there are several ensemble members with an st of a warm (large positive EFI), probably exceptional (SOT positive ~ +0.4)) but not necessarily extreme temperature anomaly, compared with SUBS-M-Climate.  Confidence in extreme temperatures rises as SOT values increase - users should focus on SOT values >0.5.


  

Moderately large positive EFI shows the equivalent rainfall anomaly distribution is above the SUBS-M-Climate anomaly distribution.  The positive upper tail SOT (quantile 90) indicates there are several ensemble members predicting extreme equivalent rainfall anomalies (above the 99th SUBS-M-Climate percentile shown by the dashed green line).  This suggests uncertainty that a significant equivalent rainfall anomaly is forecast (moderate EFI.  Note: 50% of ensemble members forecast less than about 1mm precipitation (the lower part of the SUBS-M-Climate only just above 0mm), but equally 25% of ensemble members forecast more than about 2mm precipitation (significantly above SUBS-M-Climate where about 97% of precipitation less than 2mm). If a significant rainfall occurs it could be an exceptional rainfall equivalent (SOT 0.8).  Confidence in extreme rainfall rises as SOT values increase - users should focus on SOT values >0.5.

Example of Rainfall CDF: negative SOT  

Fig8.2.4-12: The moderate positive EFI suggests the ensemble rainfall anomaly distribution is slightly above the SUBS-M-Climate anomaly distribution.  The negative upper tail SOT (quantile 90) indicates there are very few if any ensemble members predicting extreme equivalent rainfall anomalies (above the 99th SUBS-M-Climate percentile shown by the dashed green line).  This suggests uncertainty that larger than normal rainfall anomaly may be forecast (moderate EFI - but note 70% of ensemble members forecast less than about 1mm precipitation, equally 15% of ensemble members forecast more than about 2mm precipitation), but it is unlikely there will be an exceptional rainfall event (SOT -0.6).

Example of Rainfall CDF: negative SOT 

Fig8.2.4-13: An example of a rainfall CDF most frequently encountered where very few ensemble members forecast any rain at all.  The small negative EFI shows the ensemble rainfall anomaly distribution is lower than the SUBS-M-Climate anomaly distribution.  The negative upper tail SOT (quantile 90) indicates there are very few if any ensemble members (and in this case none of them) predict extreme equivalent rainfall anomalies (above the 99th SUBS-M-Climate percentile shown by the dashed green line).  This suggests confidence that larger than normal rainfall anomaly will not occur (small EFI) and an exceptional rainfall event will not occur(SOT -1.2).

Reliability diagrams in sub-seasonal range.

Reliability diagrams are available for sub-seasonal range forecasts. This gives an assessment of the current model characteristics and allows some indication of the confidence one can have in the evolution shown within the sub-seasonal ranges - unless there is good evidence to the contrary (e.g. a major change from previous forecasts in the evolution within the medium range).


Fig8.2.4-14: Illustrative examples of Reliability and ROC diagrams (here at week 1 and week 5) highlighting differences in model performance as forecasts progress.


.  

Fig8.2.4-15: Example of Cost/Loss diagrams (here at week1 and week5) illustrating two things in particular: that potential economic value for all users reduces as lead time increases, and that the range of users for whom the forecasts can have some intrinsic economic value reduces markedly as lead time increases.



See Section 12B for description of the Reliability diagrams and ROC diagrams and interpretation of Cost/Loss diagrams.



(FUG Associated with Cy49r1)