Page History

...

Clearly the error in a forecast increases through the forecast period and it is useful to have an idea of the likely magnitude and how it varies with forecast lead-time.  The accuracy of the ensemble mean (EM) can be estimated by the spread of the ensemble; on average, the larger the spread, the larger the expected EM error.  Assuming a gaussian distribution of ensemble results then the EM should also give an indication of the variability.   An analysis of the relationship between root mean square error of the EM against lead-time shows a strong similarity with a measure of the spread of the ensemble members against lead-time.  Thus the greater the spread, the greater the likely error.  On average the spread increases with lead-time, but if less than normally seen at a given lead-time then the error is likely to be less than normally expected.  The spread around the EM as a measure of theoretical accuracy applies only to the EM forecast error.  It does not apply to the median, the ensemble control (CTRL) or HRES, even if they happen to lie mid-range within the ensemble.

...

Special composite charts have been created to allow comparisons between ensemble mean (EM) and HRES.  These charts normally show great consistency from one forecast to the next and can help forecasters judge how far into the future the ENS can carry informative value for large synoptic patterns.  EM forecast values may be displayed (e.g. on ecCharts) together with the spread of the ensemble forecast values (Fig8.1.2.3).   The coloured areas do not indicate the probability of the location of a feature, but merely indicate the magnitude of the uncertainty.   Users should refer to HRES forecasts, Postage Stamp charts (example chart), Spaghetti Plot charts, or Clustering (example chart) to assess probability of departure from the EM before making forecast decisions.  

...

 Fig 8.1.2.4: An example of forecast mean sea level pressure (taken from part of an ECMWF mean and spread chart) highlighting the difference between the HRES (Green) and the ensemble mean (EM, black).  Absolute spread of ensemble members is shown by shading.  The ensemble mean is the average over all ensemble members.  It smooths the flow more in areas of large uncertainty (spread), something that cannot be achieved with a simple filtering of single forecast.  If there is large spread, the ensemble mean can be a rather weak pattern and may not represent any of the possible states.  The EM should always be used together with the spread to capture this uncertainty.  Note in particular the small depressions forecast by the HRES near 35W (shown by arrows) and the additional uncertainy within the ENS nearby suggesting at least some of the ENS members show something similar to HRES although with timing and/or location differences. 

...

ECMWF produces Mean and Spread charts and Normalised Standard Deviation charts for each ensemble run to aid understanding of the uncertainty of the forecast and whether the forecast is more or less uncertain in a given area at a given lead-time.

Fig8.1.2.5(Right): HRES PMSL (hPa) in blue and spread of the ensemble members (represented by their Standard Deviation, purple shading).  Colour scale for spread in hPa shown above the chart.   

...

The panel on the right in Fig8.1.2.5 gives an assessment of the reliability of the absolute values of the contoured ensemble mean or HRES forecast fields.  Relatively large/small absolute values of standard deviation tend to indicate relatively high/low uncertainty in forecasts.  No colouring or the paler purples imply high confidence, brighter purples/magentas imply low confidence. 

...

Fig8.1.2.6: Mean and Spread charts data time 00UTC 8 September 2017, for T+120 verifying at 00UTC 13 September 2017.

Fig8.1.2.6 Right: HRES PMSL (hPa) in blue and spread of the ensemble members (represented by their Standard Deviation, purple shading).  Colour scale for spread in hPa shown above the chart.
Fig8.1.2.6 Left: Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation (coloured shading, see Fig8.1.2.5).  Normalised Standard Deviation is a function of lead time and of geographical location.  Colour scale for Normalised Standard Deviation in hPa shown above the chart.

...

Fig8.1.2.7: Mean and Spread charts data time 00UTC 10 September 2017, for T+72 verifying at 00UTC 13 September 2017.

Fig8.1.2.7 Right: HRES PMSL (hPa) in blue and spread of the ensemble members (represented by their Standard Deviation, purple shading).  Colour scale for spread in hPa shown above the chart.  
Fig8.1.2.7 Left: Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation (coloured shading, see Fig8.1.2.5).  Normalised Standard Deviation is a function of lead time and of geographical location.  Colour scale for Normalised Standard Deviation in hPa shown above the chart.

...

• the Standard Deviation of the surface pressure pattern among the ensemble members is moderate (4hPa - 7hPa).  This implies variation (and hence uncertainty) among ensemble members regarding MSLP values in this area, or in the location of any low pressure centres.  Some ensemble members may have developed a deeper low pressure centre or sharp pressure trough in the area while others may not have; this can be resolved by inspection of the corresponding postage stamp charts.  The large standard deviation is unsurprising as one would expect variability at longer lead-times. 
• the Normalised Standard Deviation is relatively high (1·2 - 1·8).  This gives an indication of the variability among ensemble members regarding MSLP in this area compared to the variability expected at this forecast lead-time in this area.  Here there is more variability (or uncertainty) than might normally be expected, probably due to the uncertainty in the depth and movement (or even existence) of low pressure centres developed (or not) by ensemble members.
• the ensemble mean PMSL shows a broad pressure trough over northern Britain.  This probably relates to the large normalised spread; it is likely that some ensemble members also have this feature.  HRES shows development of a fairly deep depression (~987hPa) but HRES should only be considered as one member of the ENS and has low weighting at T+120. 

...

• the Standard Deviation of the surface pressure pattern among the ensemble members is moderate (4hPa - 7hPa) but of less spatial extent than seen at T+120 (Fig8.1.2.6).  This implies less widespread variation (and hence uncertainty) among ensemble members in this area regarding MSLP values or location of any low pressure centres, although the detail of any low pressure centre or trough and/or its location is imprecise.
• the Normalised Standard Deviation is much greater (2·5 - 5·0) than seen at T+120 (Fig8.1.2.6).  This implies variability among ensemble members is significantly higher in this area regarding MSLP compared to the variability expected at this forecast lead-time.  Here this is probably due to the depth and movement of possibly deeper low pressure centre(s) developed by ensemble members.
• the ensemble mean PMSL shows a sharp pressure trough (sharper than at T+120 (Fig8.1.2.6)) over northern Britain, and the large standard deviation suggests some ensemble members develop a low pressure centre or sharp pressure trough in the area.  However some ensemble members may not develop any low pressure at all.  This can be resolved by reference to the corresponding postage stamp charts.  HRES shows development of a rather deeper and more vigorous depression (~983hPa) (deeper than at T+120 (Fig8.1.2.6)).  This is supported by HRES, and although HRES should only be considered as one member of the ENS, it has a higher weighting at T+72 than at T+120.

...

• Variability within the ensemble members (as measured by Standard Deviation) usually can be expected to increase as forecast lead-times increase.
• When HRES is used in combination with ensemble forecasts, the weighting of the HRES decreases as the lead time increases and HRES may be used with less confidence.
• Large Normalised Standard Deviation states only that the variability of the ensemble members is greater than expected at this forecast lead-time and location.  It does not necessarily imply greater uncertainty.  One would anyway expect greater variability in ensemble results in the vicinity of a forecasted deep depression.

_{Updated/Amended 24/10/20 - amended chart links to open access.}