Page History

...

Ensemble mean and ensemble spread

Ensemble

...

mean

The ensemble mean is the average of the forecast values of the ensemble members at a given forecast time (i.e. the sum of the values divided by the number of ensemble members).  The mean leans towards the values of a greater number of ensemble members and less weight is given to outliers.  It is mostly used with medium range forecasts where the mean leans tends towards the most probable value.

Ensemble Median

The ensemble mean is most suited to parameters like temperature and pressure because these usually have rather symmetric Gaussian distributions.

Ensemble median

The ensemble The median is the middle value of the forecast values of the ensemble members at a given forecast time when sorted into a list (i.e. the same amount number of ensemble member values below and above the middle value).  The median lies at the centre of the range of the ensemble members and can be more descriptive of the data set than the mean.  It is mostly used with seasonal forecasts where the range of values can be quite large.

Ensemble Spread

The ensemble median is is more suited to parameters like wind speeds and precipitation because these usually have skewed distributions.  

Ensemble spread and Standard Deviation

The ensemble spread is a measure of the difference of a forecast between the ensemble members.

The ensemble spread is spread is a measure of the difference between the members and is represented by the standard deviation (Std) with respect to the ensemble mean (EM).  On average, small spread theoretically

Theoretically, on average:

• small spread or low standard deviation indicates high forecast accuracy

...

• large spread

...

• or high standard deviation indicates low forecast accuracy.     

The ensemble spread should reflect the diversity of all possible outcomes.

The .  The ensemble spread is flow-dependent and varies for different parameters.  It usually increases with the forecast range, but there can be cases when the spread is larger at shorter forecast lead times than at longer.  This might happen when the first days are characterized by strong synoptic systems with complex structures but are followed by large-scale fair weather high-pressure systems.  The ensemble spread should reflect the diversity of all possible outcomes, in particular when the deterministic forecasts are “

Two similar-looking forecast charts may display large differences in geopotential if they contain systems with strong gradients that are jumpy”, which might indicate that very different weather developments are possible.  Two similar-looking forecast charts may display large differences in geopotential if they contain systems with strong gradients that are slightly out of phase.  Conversely, two synoptically rather different forecast charts will display small differences if the gradients are weak.  The spread refers to the uncertainty of the values of mean sea level pressure, geopotential height, wind or temperature, but not necessarily to the flow patterns.  Also “jumpy”, deterministic forecasts might indicate that very different weather developments are possible.   Such aspects are reflected in charts of ensemble spread.   

Relationship

...

of ensemble mean and ensemble spread against forecast lead-time

The ensemble mean (or, on occasion, the ensemble median) forecast tends to average out the less predictable atmospheric scales.   As the forecast proceeds the variation between the results of ENS ensemble members gradually increases.  The ensemble mean, of course, will lie within the envelope of ensemble members throughout the forecast.     

Fig8.1.2.-1: The diagram is a schematic plume showing the relation between the standard deviation of the ensemble members for the whole forecast range (shaded area), an individual forecast (green line), the ensemble control member (CTRL, blue line) and the ensemble mean (EM, red line).  The EM ensemble mean lies more or less in the middle of the ensemble spread whereas any individual ensemble member (green line), can lie anywhere within the spread.  The CTRLensemble control, which does not constitute a part of the plume, can even on rare occasions (theoretically on average 4% of the time) be outside the standard deviation plume.

Clearly the error in a forecast increases through the forecast period and it It is useful to have an idea of the likely magnitude of the forecast error and how it varies with forecast lead-time.   The accuracy of the ensemble mean (EM) can be estimated by the spread of the ensemble; on .

On average, the larger the spread , the implies larger the expected EM error of the ensemble mean.  Assuming If one assumes a gaussian distribution of ensemble results then the EM ensemble mean should also give an indication of the variability.

   An analysis of the relationship between root mean square error of the EM ensemble mean against lead-time shows a strong similarity with to a measure of the spread of the ensemble members against lead-time.   Thus  Thus the greater the spread, the greater the likely error. 

On average, the spread increases with lead-time, but if less than normally seen at a given lead-time then the error is likely to be less than normally expected.  

The spread around the EM ensemble mean as a measure of theoretical accuracy applies only to the EM ensemble mean forecast error.  It does not apply to the median , or the ensemble control (CTRL) or HRES, even if they happen to lie mid-range within the ensemble.

Fig8.1.2.-2: The graph shows the error, on average, in 850hPa temperature for the extra-tropical Northern Hemisphere extratropics at at various forecast lead-lead times. The  The relationship, on average, between ENS ensemble root mean square error (full line) and ENS ensemble spread (dashed line), shows a strong correlation.   A low (or high) spread in the forecast, on average, implies low (or high) error on average (though at the same time any individual EM ensemble mean forecast may by chance be good or bad).

Mean and

...

spread charts

Special composite charts have been created to allow comparisons between the ensemble mean (EM) and HRES.  These charts normally show great consistency from one forecast to the next and can help forecasters judge how far into the future the ENS can carry informative value for large synoptic patterns.  EM forecast values may be displayed (e.g. on ecCharts) together with the spread of the ensemble forecast values (Fig8.1.2.3).   The coloured areas do not indicate the probability of the location of and the ensemble control (e.g. on ecCharts) (Fig8.1.2-3).  The coloured areas do not indicate the probability of the location of a feature, but merely indicate the magnitude of the uncertainty.   Users should refer to HRES forecasts, Postage Stamp charts (example chart), Spaghetti Plot charts, or Clustering (example chart) to assess probability of departure from the EM ensemble mean before making forecast decisions.  

Fig8.1.2.-3: 500hPa ENS ensemble mean geopotential height (in red, 580dam isopleth crosses east Italy and north Greece) and spread of geopotential height among ensemble members (coloured according to the scale).  Forecast for DT 12UTC 13 August 8 Aug 2017, T+120 from ENS data time 12UTC 8 Aug VT 12UTC 13 August 2017.  The greatest spread shows the areas of greatest uncertainty.  The light green area indicates a spread of 4-5dam and this could be:

• where geopotential heights on many ENS ensemble members are lower than the ENS ensemble mean (with a few ENS ensemble members well above the ENS ensemble mean) suggesting the trough will probably be slower or broader.
• where geopotential heights on many ENS ensemble members are higher than the ENS ensemble mean (with a few ENS ensemble members well below the ENS ensemble mean) suggesting the trough will probably be faster or sharper.

 Fig 8Fig8.1.2.-4: An example of forecast mean sea level pressure (taken from part of an ECMWF mean and spread chart) highlighting the difference between the HRES ensemble control (Greengreen) and the ensemble mean (EM, black).  Absolute spread of ensemble members is shown by shading.  The ensemble mean is the average over all ensemble members.  It smooths the flow more in areas of large uncertainty (large spread), something that cannot be achieved with a simple filtering of single forecast.  If there is large spread, the ensemble mean can be a rather weak pattern and may not represent any of the possible states.  The EM ensemble mean should always be used together with the spread to capture this uncertainty.  Note in particular the small depressions forecast by the HRES ensemble control near 35W (shown by arrows) and the additional uncertainy uncertainty (darker purple) within the ENS nearby suggesting ensemble members nearby.  This suggests at least some of the ENS ensemble members show something similar to HRES although the ensemble control although with timing and/or location differences. 

The

...

normalised standard deviation

The ensemble spread tends to show a strong geographical dependence.  For geopotential Geopotential and pressure there is generally show little spread at low latitudes, but variability .  Variability is greater at mid-latitudes and the spread is consequently rather higher.  This latitude dependence tends to obscure the features of a given situation and a normalised spread or standard deviation (Nstd) is more useful.  For this, the spread, measured by the standard deviation (Std) of ensemble member values at a given point and lead time, is normalised against the mean of the spread of the 30 most recent 00UTC ENS ensemble members (Mstd) for 00UTC runs (or 12UTC ENS ensemble members (Mstd) for 30 most recent 12UTC ENS runs) for the same lead-times and geographical locations.

...

      Nstd is the Normalised Standard Deviation.
      Std is the Standard Deviation of the latest ENSensemble.
      Mstd is the Mean Standard Deviation of the spread of the 30 most recent 00UTC or 12UTC ENS ensemble runs.

The Normalised Standard Deviation highlights geographical areas of unusually high or low spread, where the uncertainty is larger or smaller than over the last 30 days.  If the spread in a particular area, an Nstd value: 

• near 1 implies the spread remains similar to previous spreads in that area, irrespective of whether the spread is large or small.
• >1 implies the spread is greater than recently. 
• <1 implies the spread is greater than recently. 

then Nstd has a value near 1, irrespective of whether the spread is large or small.  If it has greater spread that recently then Nstd >1, if it has less spread than recently then Nstd <1.  The normalised spread shows the increase or decrease in spread at a location, .  It does not show the magnitude of the spread, and therefore .  Therefore it highlights relatively low or relatively high uncertainty, but not the uncertainty itself.

ECMWF produces Mean and Spread charts and Normalised Standard Deviation charts for each ensemble run to aid understanding of the uncertainty of the forecast and whether the forecast is more or less uncertain in a given area at a given lead-time.

Fig8.1.2.-5(Right): HRES Ensemble control PMSL (hPa) in blue and spread of the ensemble members (represented by their Standard Deviationthe standard deviation, purple shading).  Colour scale for spread in hPa shown above the chart.   

Fig8.1.2.-5(Left): Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation normalised standard deviation (coloured shading).  Normalised Standard Deviation standard deviation (Nstd) is calculated by dividing the Standard Deviation (standard deviation (Std) (in right hand frame above) by a Mean Standard Deviation, mean standard deviation (Mstd), which is a pre-computed mean of the standard deviations of the 30 most recent 00UTC (or 12UTC) ensemble forecastsfor forecasts for the given lead time (this is also a function of location).  Colour scale for Normalised Standard Deviation normalised standard deviation in hPa shown above the chart - uncoloured indicates a similarity with previous ENS ensemble mean values.

The panel on the right in Fig8.1.2.-5 gives an assessment of the reliability of the absolute values of the contoured ensemble mean or HRES forecast mean forecast fields.  Relatively large/small absolute values of standard deviation tend to indicate relatively high/low uncertainty in forecasts.  No colouring or the paler purples imply high higher confidence, brighter purples/magentas imply low lower confidence. 

The panel on the left in Fig8.1.2.-5, shows the normalised standard deviation , which aims to put the standard deviation measure into the context of the general ensemble behaviour within the chart area over the last 30 days.   It tells whether the most recent ensemble is showing greater or less spread (and hence uncertainty) than recent ensemble results.  If the spread at Day5 of .

For a particular set of ensemble forecasts (right panel) in a certain area at day5, if the spread:

• is similar to the spread that had recently been seen there at day5, then the shading of the normalised standard deviation (left panel) has a value near 1 (uncoloured).
• seems to be large, but lately has

...

• also been similarly large at day5, then the shading of the normalised standard deviation (left panel) has a value close to 1 (uncoloured)

...

• :
• is greater than the spread that had recently been seen there at day5, then the shading of the normalised standard deviation (left panel) indicates a value rather greater than 1 (purple shading).
• is less than the spread that had recently been seen there at

...

• day5, then the shading of the normalised standard deviation (left panel) indicates a value rather

...

• lower than 1 (

...

• green shading).

  So although the forecast for (say) longer lead-times in the ENS ensemble (say days 8-10) will usually be of rather low confidence, there will be some occasions when one can be rather more confident than usual for this lead-time.  The normalised standard deviation will tend to show this by green shading.

An

...

example of an

...

analysis of

...

ensemble mean and spread charts

Comparing the run-to-run changes in Mean ensemble mean and Spread spread charts and the Normalised Standard Deviation normalised standard deviation charts can be informative and aid an assessment of confidence in the forecast.

Fig8.1.2.-6: Mean and Spread charts data time DT 00UTC 8 September 2017, for T+120 verifying at VT 00UTC 13 September 2017.

Fig8.1.2.-6(Right): Ensemble control HRES PMSL (hPa) in blue and spread of the ensemble members (represented by their Standard Deviationthe standard deviation, purple shading).  Colour scale for spread in hPa shown above the chart.
Fig8.1.2.-6(Left): Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation normalised standard deviation (coloured shading, see Fig8.1.2.-5).  Normalised Standard Deviation standard deviation is a function of lead time and of geographical location.  Colour scale for Normalised Standard Deviation normalised standard deviation in hPa shown above the chart.

Fig8.1.2.-7: Mean and Spread charts data time DT 00UTC 10 September 2017, for T+72 verifying at VT 00UTC 13 September 2017.

Fig8.1.2.-7(Right): Ensemble control HRES PMSL (hPa) in blue and spread of the ensemble members (represented by their the Standard Deviation, purple shading).  Colour scale for spread in hPa shown above the chart.  
Fig8.1.2.-7(Left): Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation (coloured shading, see Fig8.1.2.-5).  Normalised Standard Deviation is a function of lead time and of geographical location.  Colour scale for Normalised Standard Deviation in hPa shown above the chart.

...

Consider the charts for T+120 (Fig8.1.2.-6) and T+72 (Fig8.1.2.-7), both verifying at 00UTC 13 September 2017.

Over Scotland and northern England at T+120 (Fig8.1.2.-6):

• the Standard Deviation standard deviation of the surface pressure pattern among the ensemble members is moderate (4hPa - 7hPa).  This implies some variation (and hence uncertainty) among ensemble members regarding MSLP values in this area, or in the location of any low pressure centres.  Some ensemble members may have developed a deeper low pressure centre or sharp pressure trough in the area while others may not have; this can be resolved by inspection of the corresponding postage stamp charts.  The large standard deviation is unsurprising as one would expect variability at longer lead-times. 
• the Normalised Standard Deviation normalised standard deviation is relatively high (1·2 - 1·8).  This gives an indication of the variability among ensemble members regarding MSLP in this area compared to the variability expected at this forecast lead-time in this area.  Here there is more variability (or uncertainty) than might normally be expectedfrom recent ensemble forecasts, probably due to the uncertainty in the depth and movement (or even existence) of low pressure centres developed (or not) by ensemble members.
• the ensemble mean PMSL shows a broad pressure trough over northern Britain.  This probably relates to the large normalised spread; it is likely that some ensemble members also have this feature.   HRES shows  Ensemble control shows development of a fairly deep depression (~987hPa) but HRES the ensemble control should only be considered as one member of the ENS and has low weighting at T+120ensemble. 

Over Scotland and northern England at T+72 (Fig8.1.2.-7):

• the Standard Deviation standard deviation of the surface pressure pattern among the ensemble members is moderate (4hPa - 7hPa) but of less spatial spacial extent than seen at T+120 (Fig8.1.2.-6).  This implies less widespread variation (and hence uncertainty) among ensemble members in this area regarding MSLP values or location of any low pressure centres, although the detail of any low pressure centre or trough and/or its location is imprecise.
• the Normalised Standard Deviation normalised standard deviation is much greater (2·5 - 5·0) than seen at T+120 (Fig8.1.2.-6).  This implies variability among ensemble members is significantly higher in this area regarding MSLP compared to the variability expected at this forecast lead-time.  Here this This is probably due to the depth and movement of possibly deeper low pressure centre(s) developed by ensemble members.
• the ensemble mean PMSL shows a sharp pressure trough (sharper than at T+120 (Fig8.1.2.-6)) over northern Britain, and the .  The large standard deviation suggests some ensemble members develop a low pressure centre or sharp pressure trough in the area.  However some ensemble members may not develop any low pressure at all.  This can be resolved by reference to the corresponding postage stamp charts.   HRES shows  The ensemble control shows development of a rather deeper and more vigorous depression (~983hPa) (deeper than at T+120 (Fig8.1.2.-6)).   This is supported by HRES, and although HRES  However, the ensemble control should only be considered as one member of the ENS, it has a higher weighting at T+72 than at T+120.ensemble. 

Elsewhere, comparing the charts for T+120 (Fig8.1.2.-6) and for T+72 (Fig8.1.2.-7):

• the the Standard Deviation standard deviation of the ensemble surface pressure patterns is significantly less at T+72 than at T+120.
• the Normalised Standard Deviation normalised standard deviation of the ensemble surface pressure pattern in mid-Atlantic:
  • at T+72 green area indicating less spread than recent forecasts in this area.
  • at T+120 white colouring indicating spread similar to recent forecasts in this area.
• the ensemble mean PMSL shows only small differences.

...

• Variability within the ensemble members (as measured by Standard Deviationstandard deviation) usually can be expected to increase as forecast lead-times increase.
• When HRES is used in combination with ensemble forecasts, the weighting of the HRES decreases as the lead time increases and HRES may be used with less confidence.
• Large Normalised Standard Deviation Large normalised standard deviation states only that the variability (or uncertainty) of the ensemble members is greater than expected more than from recent ensemble forecasts at this forecast lead-time and location.  It does not necessarily imply greater uncertainty.  One would anyway expect greater variability in ensemble results in the vicinity of a forecasted deep depression.

...