ENS Mean and Spread
The ensemble spread is a measure of the difference between the members and is represented by the standard deviation (Std) with respect to the ensemble mean. On average, small spread indicates high theoretical forecast accuracy, large spread indicates low theoretical forecast accuracy. The ensemble spread is flow-dependent and varies for different parameters. It usually increases with the forecast range, but there can be cases when the spread is larger at shorter forecast ranges than at longer. This might happen when the first days are characterized by strong synoptic systems with complex structures but are followed by large-scale fair weather high-pressure systems. The ensemble spread should reflect the diversity of all possible outcomes, in particular when the deterministic forecasts are “jumpy”, which might indicate that very different weather developments are possible. Two similar-looking forecast charts may display large differences in geopotential if they contain systems with strong gradients that are slightly out of phase. Conversely, two synoptically rather different forecast charts will display small differences if the gradients are weak. The spread refers to the uncertainty of the values of mean sea level pressure, geopotential height, wind or temperature, but not necessarily to the flow patterns. Such aspects are reflected in charts of ensemble spread.
Relationship between Ensemble Mean and Ensemble Spread against Forecast Lead-time
The ENS provides the ensemble mean (or the ensemble median) forecast which tends to average out the less predictable atmospheric scales. As the forecast proceeds the variation between the results of ENS members gradually increases. The ENS mean of course will lie within the envelope of ENS members throughout the forecast.
Clearly the error in a forecast increases through the forecast period and it is useful to have an idea of the likely magnitude and how it varies with forecast lead-time. The accuracy of the EM can be estimated by the spread of the ensemble: on average, the larger the spread, the larger the expected EM error. Assuming a gaussian distribution of ensemble results then the ensemble mean should also give an indication of the variability. An analysis of the relationship between root mean square error of the ENS mean against lead-time shows a strong similarity with a measure of the spread of the ENS against lead-time. Thus the greater the spread, the greater the likely error. On average the spread increases with lead-time, but if less than normally seen at a given lead-time then the error is likely to be less than normally expected.
Fig184.108.40.206: The graph shows the error, on average, in 850hPa temperature for the Northern Hemisphere extratropics at various forecast-lead times. The relationship, on average, between ENS root mean square error (full line) and ENS spread (dashed line), shows a strong correlation. A low (or high) spread in the forecast implies low (or high) error on average (though at the same time any individual EM forecast may by chance be good or bad).
Special composite charts have been created to facilitate comparisons between ensemble mean and HRES. These charts normally show great consistency from one forecast to the next and can help forecasters judge how far into the future the ENS can carry informative value for large synoptic patterns. Ensemble mean forecast values may be displayed (e.g. ecCharts) together with the ENS spread of the ensemble forecast values (Fig220.127.116.11). The coloured areas do not indicate the probability of location of a feature, but merely indicate the magnitude of the uncertainty. Users should refer to HRES forecasts, Postage Stamp charts, Spaghetti Plot charts, or Clustering to assess probability of departure from the ENS mean before making forecast decisions.
Fig18.104.22.168: 500hPa ENS mean geopotential height (in red, 580dam isopleth crossing east Italy and north Greece) and spread of geopotential height among ENS members (coloured according to the scale). Forecast for 12UTC 13 August 2017 T+120 from ENS data time 12UTC 8 Aug 2017. The greatest spread shows the areas of greatest uncertainty. The light green area indicates a spread of 4-5dam and this could be:
- where geopotential heights on many ENS members are lower than the ENS mean (with a few ENS members well above the ENS mean) suggesting the trough will probably be slower or broader.
- where geopotential heights on many ENS members are higher than the ENS mean (with a few ENS members well below the ENS mean) suggesting the trough will probably be faster or sharper.
Fig 22.214.171.124: An example of forecast mean sea level pressure (taken from part of an ECMWF mean and spread chart) highlighting the difference between the HRES (Green) and the ENS Mean (black). Absolute ENS spread is shown by shading. The ensemble mean is the average over all ensemble members. It smooths the flow more in areas of large uncertainty (spread), something that cannot be achieved with a simple filtering of single forecast. If there is large spread, the ensemble mean can be a rather weak pattern and may not represent any of the possible states. The ensemble mean should always be used together with the spred to capture this uncertainty. Note in particular the small depressions forecast by the HRES near 35W (shown by arrows) and the additional uncertainy within the ENS nearby suggesting at least some of the ENS members show something similar to HRES although with timing and/or location differences.
The Normalised Standard Deviation
The ensemble spread tends to show a strong geographical dependence. For geopotential and pressure there is generally little spread at low latitudes, but variability is greater at mid-latitudes and the spread is consequently rather higher. This latitude dependence tends to obscure the features of a given situation and a normalised spread (Nstd) is more useful. For this, the spread, measured by the standard deviation (Std) of ENS values at a given point and lead time, is normalised against the mean of the spread of the 30 most recent 00UTC ENS (Mstd) for 00UTC runs (or 12UTC ENS (Mstd) for 30 most recent 12UTC ENS runs) for the same lead-times and geographical locations.
The Normalised Spread is defined as: Nstd = Std/Mstd
where, for a given forecast lead-time and location:
Nstd is the Normalised Standard Deviation.
Std is the Standard Deviation of the latest ENS.
Mstd is the Mean Standard Deviation of the spread of the 30 most recent 00UTC or 12UTC ENS runs.
The Normalised Standard Deviation highlights geographical areas of unusually high or low spread, where the uncertainty is larger or smaller than over the last 30 days. If the spread in a particular area remains similar to previous spreads in that area then Nstd has a value near 1, irrespective of whether the spread is large or small. If it has greater spread that recently then Nstd >1, if it has less spread than recently then Nstd <1. The normalised spread shows the increase or decrease in spread at a location, not the magnitude of the spread, and therefore highlights relatively low or relatively high uncertainty, not the uncertainty itself.
ECMWF produces Mean and Spread charts and Normalised Standard Deviation charts for each ensemble run to aid understanding of the uncertainty of the forecast and whether the forecast is more or less uncertain in a given area at a given lead-time.
Fig126.96.36.199(Right): HRES PMSL (hPa) in blue with spread of the ENS members represented as the Standard Deviation of the ENS members (purple shading). Colour scale in hPa shown above the chart.
Fig188.8.131.52(Left): Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation (coloured shading). Normalised Standard Deviation is calculated by dividing the Standard Deviation (right hand frame above) by a Mean Standard Deviation, a pre-computed mean of the standard deviations of the 30 most recent 00UTC (or 12UTC) ensemble forecastsfor the given lead time (this is also a function of location). Colour scale in hPa shown above the chart - uncoloured indicates a similarity with previous ENS mean values.
The panel on the right in Fig184.108.40.206 gives an assessment of the reliability of the absolute values of the contoured ENS Mean or HRES forecast fields. Relatively large/small absolute values of standard deviation tend to indicate relatively high/low uncertainty in forecasts. No colouring or the paler purples imply high confidence, brighter purples/magentas imply low confidence.
The panel on the left in Fig220.127.116.11, the normalised standard deviation aims to put the standard deviation measure into the context of the general ensemble behaviour, in that area, over the last 30 days. It tells whether the most recent ENS is showing greater or less spread (and hence uncertainty) than recent ENS results. If the spread at Day5 of a particular set of ensemble forecasts (right panel) seems to be large, but has of late also tended to be equally large at Day5 in the same area, then the left panel shading will denote a value that is close to 1 (uncoloured). If the spread in a particular area at Day5 in one ensemble is greater/less than the spread that had recently been seen there at Day5, then the shading of the normalised standard deviation (left panel) indicates a value rather greater/less than 1 (purple/green shading). So although the forecast for (say) longer lead-times in the ENS (say days 8-10) will usually be of rather low confidence, there will be some occasions when one can be rather more confident than usual for this lead-time. The normalised standard deviation will tend to show this by green shading.
An Example of an Analysis of Mean and Spread Charts
Comparing the run-to-run changes in Mean and Spread charts and the Normalised Standard Deviation charts can be informative and aid an assessment of confidence in the forecast.
Fig18.104.22.168: Mean and Spread charts data time 00UTC 8 September 2017, for T+120 verifying at 00UTC 13 September 2017.
Fig22.214.171.124 Right: HRES PMSL (hPa) in blue with spread of the ENS members represented as the Standard Deviation of the ENS members (purple shading). Colour scale in hPa shown above the chart.
Fig126.96.36.199 Left: Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation (coloured shading, see Fig188.8.131.52). Normalised Standard Deviation is a function of lead time and of geographical location. Colour scale in hPa shown above the chart.
Fig184.108.40.206: Mean and Spread charts data time 00UTC 10 September 2017, for T+72 verifying at 00UTC 13 September 2017.
Fig220.127.116.11 Right: HRES PMSL (hPa) in blue with spread of the ENS members represented as the Standard Deviation of the ENS members (purple shading). Colour scale in hPa shown above the chart.
Fig18.104.22.168 Left: Ensemble mean PMSL (hPa) in blue with Normalised Standard Deviation (coloured shading, see Fig22.214.171.124). Normalised Standard Deviation is a function of lead time and of geographical location. Colour scale in hPa shown above the chart.
Consider the charts for T+120 (Fig126.96.36.199) and T+72 (Fig188.8.131.52), both verifying at 00UTC 13 September 2017.
Over Scotland and northern England at T+120 (Fig184.108.40.206):
- the Standard Deviation of the ENS surface pressure pattern is moderate (4hPa - 7hPa). This implies variation (and hence uncertainty) among ENS members regarding MSLP values in this area, or the location of any low pressure centres. Some ENS members may have developed a deeper low pressure centre or sharp pressure trough in the area while others may not have; this can be resolved by inspection of the corresponding postage stamp charts. The large standard deviation is unsurprising as one would expect variability at longer lead-times.
- the Normalised Standard Deviation is relatively high (1·2 - 1·8). This gives an indication of the variability among ENS members regarding MSLP in this area compared to the variability expected at this forecast lead-time in this area. Here there is more variability (or uncertainty) than might normally be expected, probably due to the uncertainty in the depth and movement (or even existence) of low pressure centres developed (or not) by ENS members.
- the ENS mean PMSL shows a broad pressure trough over northern Britain. This probably relates to the large normalised spread; it is likely that some ENS members also have this feature. HRES shows development of a fairly deep depression (~987hPa) but HRES should only be considered as one member of the ENS and has low weighting at T+120.
Over Scotland and northern England at T+72 (Fig220.127.116.11):
- the Standard Deviation of the ENS surface pressure pattern remains moderate (4hPa - 7hPa) but of less spatial extent. This implies less widespread variation (and hence uncertainty) among ENS members in this area regarding MSLP values or location of any low pressure centres, although the detail of any low pressure centre or trough and/or its location remains imprecise.
- the Normalised Standard Deviation has become much greater (2·5 - 5·0). This implies a significant increase in variability among ENS members in this area regarding MSLP compared to the variability expected at this forecast lead-time in this area. Here this is probably due to the depth and movement of possibly deeper low pressure centre(s) developed by ENS members.
- the ENS mean PMSL shows a sharp pressure trough (sharper than Fig18.104.22.168 shows at T+120) over northern Britain, and the large standard deviation suggests some ENS members develop a low pressure centre or sharp pressure trough in the area. However some ENS may not develop ant low pressure at all. This can be resolved by reference to the corresponding postage stamp charts. HRES shows development of a rather deeper and more vigorous depression (~983hPa) (deeper than Fig22.214.171.124 shows at T+120). This is supported by HRES, and although HRES should only be considered as one member of the ENS, it has a higher weighting at T+72 than at T+120.
Elsewhere, comparing the charts for T+120 (Fig126.96.36.199) and for T+72 (Fig188.8.131.52):
- the the Standard Deviation of the ENS surface pressure pattern has reduced significantly
- the Normalised Standard Deviation of the ENS surface pressure pattern has reduced and becomes very small in mid-Atlantic (green area).
- the ENS mean PMSL shows only small differences.
- The variability within the ENS (as measured by Standard Deviation) usually can be expected to decrease with shorter forecast lead-time.
- When HRES is used in combination with ENS forecasts, the weighting of the HRES will be increasing as the lead time reduces and may be used with more confidence.
- Larger Normalised Standard Deviation states only that the variability of the ENS is greater than expected at this forecast lead-time in an area. It does not necessarily imply greater uncertainty. One would anyway expect greater variability in ENS results in the vicinity of a forecasted deep depression.