image © AzriSuratmin/iStock/Thinkstock
Rationale of the Forecast Ensemble - ENS
Uncertainties in NWP forecasting
No NWP model can produce consistently or precisely correct forecasts and there must be some uncertainty in the results of each forecast. The value of NWP forecasts that are produced would be greatly enhanced if the quality of those forecasts could be assessed beforehand. Consequently methods have been, and continue to be, developed to provide advance knowledge on the certainty (or uncertainty) of a particular forecast, and what possible alternative developments might occur. This is in parallel with improving the observational network, the data assimilation system, and the NWP models themselves.
The ECMWF forecast ensemble is based upon the idea that incorrect forecasts result from a combination of initial analysis errors and model deficiencies, the former dominating during the first five days or so.
Forecast models can produce incorrect results because of:
- initial condition uncertainties (also originating from errors in the first guess forecast):
- Lack of observations.
- Observation error.
- Errors in the data assimilation.
- model uncertainties:
- Limited resolution.
- Weaknesses in parameterisation of physical processes.
- boundary condition uncertainties:
- Insufficient detail of sub-grid scale orography.
- Insufficient knowledge of changing surface characteristics.
- Parameterised derivation of surface fluxes.
- the chaotic nature of the atmosphere:
- Small uncertainties grow to large errors (unstable flow)
- Small-scale errors will affect the large-scale (non-linear dynamics)
- Error-growth is flow dependant.
But even very good analysis systems and forecast models are prone to errors.
Analysis errors amplify most easily where the the atmosphere is most sensitive to small differences, in particular where strong baroclinic systems develop. These errors then move downstream and further amplify or change, and thereby affect the large-scale flow.
Structure and operation of the ENS
To estimate the effect of possible initial analysis errors and the consequent uncertainty of the forecasts, an ensemble is formed of many (currently 50) different, “perturbed” initial states and one unperturbed analysis (the control member (CTRL)). The different perturbations are derived at analysis time during the generation of the ensemble.
The ENS forecast suite is then run using each of the perturbed and the unperturbed analyses as a starting point giving a range of forecast results which may diverge radically or remain broadly similar. To deal with uncertainty in the structure of the parameterisation schemes or with errors due to incomplete IFS modelling of unresolved scales, etc., perturbations are continually inserted into the ensemble members (but not the ensemble control) throughout execution of the forecasts. The perturbations are supplied by the Stochastically Perturbed Parameterisation Tendencies Scheme (SPPT). The ensemble control member (CTRL) is unperturbed and does not use these "stochastic physics" perturbations during execution.
Singular Vectors (SVs) pick up localised areas of strong barotropic and baroclinic instability and these are also supplied as perturbations to the ensemble members.
Processing the ensemble of forecasts is computationally expensive and, in order to save computation time, the ensemble members are run with a lower resolution than the HRES (currently 18km compared to 9km for HRES).
Qualitative use of the ENS
If forecasts starting from these perturbed analyses more or less agree with the CTRL forecast, then the atmosphere can be considered to be in a predictable state and any of the expected analysis errors would not have a significant impact. In such a case, it would be possible to issue a categorical forecast with reasonable, but not total, certainty. On the other hand, if the perturbed forecasts deviate significantly from the CTRL forecast and from each other, it can be concluded that the atmosphere is in a rather unpredictable state. In this case, it would not be possible to issue a categorical forecast with any certainty. The way in which the perturbed forecasts differ from each other provides valuable indications of which weather patterns are likely to develop or, often equally importantly, not develop. Guidance on how to best exploit the relative strengths of deterministic and probabilistic output, on how to deal with uncertainty and on aids to interpretation of various ENS outputs are given elsewhere within the this User Guide.
Fig5.1: An ensemble of forecasts produces a range of possible scenarios rather than a single predicted value. Addition or subtraction of small perturbations to the initial distribution of a parameter give several (50 at ECMWF) equiprobable ensemble members each slightly different from the initial, unperturbed analysis of the control member (the 51st member of the ensemble). The ensemble members evolve through the forecast period in slightly different ways and the distribution of the ensemble members gives an indication of the likelihood of occurrence of the different scenarios. In the schematic diagram, compared to initial conditions, a few ensemble members (small peak in ENS numbers) are grouped to give a similar forecast that is cooler, two forecasts predict relatively much warmer conditions, but the majority show a modest warming over initial conditions. Each solution is possible, the solution associated with the larger peak (larger number of ENS members) is more probable.
Quantitative use of the ENS
The ensemble mean (EM) forecast, or if required the ensemble median forecast (not necessarily the same as the EM) can be calculated from the ENS. This tends to average out the less predictable atmospheric scales. The accuracy of the EM can be estimated theoretically by the spread of the ensemble so that, on average, the expected EM error is proportional to ENS spread. More importantly, the ENS provides information from which the probability of alternative developments is calculated, in particular those related to risk of extreme or high-impact weather.
The ensemble spread is a measure of the difference between the members and is represented by the standard deviation (Std) with respect to the EM. On average, small spread indicates high forecast accuracy of the ensemble mean, and indeed of the ensemble members in general; larger spread corresponds to lower forecast accuracy of the ensemble mean, and of most of the ensemble members. The ensemble spread is flow-dependent and in relative terms will vary for different parameters (e.g. in winter anticyclonic conditions over land spread might be relatively high for 2m temperature, but relatively low for mean sea level pressure). Spread usually increases with the forecast range, but there can be cases when the spread is larger at shorter forecast ranges than at longer ranges. This might happen when the first days are characterized by strong synoptic systems with complex structures but are followed by large-scale “fair weather” high pressure systems.
The spread around the EM as a measure of accuracy applies only to the EM forecast error, and not to the median, nor the Control, nor HRES, even if they happen to lie mid-range within the ensemble. The spread of the ensemble, relative to a particular ensemble member is, for example, about 41% larger than the spread around the EM. The spread with respect to the CTRL is initially the same as for the EM, but gradually increases, ultimately reaching the same 41% excess as any member (see Fig5.2).
Fig5.2: The diagram shows schematically the relation between the spread of the ensemble for the whole forecast range (orange shaded area). The EM (red line) lies in the middle of the ensemble spread whereas any individual ensemble member (blue line), can lie anywhere within the spread. The Control (green line), which does not constitute a part of the plume, can even on rare occasions (theoretically on average 4% of the time) be outside the plume.
Characteristics of a good ensemble
Forecasts from a good ensemble should:
- display no mean errors (bias); otherwise the probabilities will be biased as well.
- exhibit sharpness (i.e. have relatively small spread where the uncertainty is small).
- have the ability to span the full climatological range; otherwise the probabilities will either over- or under-forecast the risks of anomalous or extreme weather events.
Additional Sources of Information
(Note: In older material there may be references to issues that have subsequently been addressed)
- Watch a comprehensive lecture on uncertainty and the use of an ensemble prediction system giving insight to forecast uncertainties, the use of an ensemble prediction system, and the ECMWF ENS system.
- Watch training videos on sources of uncertainty and quantifying uncertainty.
- Read about EDA and use of EDA and SV based perturbations in the EPS (pages 17-28).
- View ECMWF eLearning module on ensemble forecasting.