Forecast Error Growth

Relationship between Forecast Range and Forecast Error

Forecast error growth is, on average, largest at the beginning of the forecast.  At longer forecast ranges it levels off asymptotically towards the error level of persistence forecasts, pure guesses or the difference between two randomly chosen atmospheric states (see Fig4.1-1).  This error level is significantly higher than the average error level for a simple climatological average if used as a forecast.  Forecast verification is discussed in the annexe.


Fig4.1-1:  A schematic illustration of the forecast error development of a state-of-the-art NWP (full curve), persistence and guesses (dotted curve), whose errors converge to a higher error saturation level than modified forecasts, which converge at a lower RMSE level (dashed curve). 


Relationship between Scale and Predictive Skill

It is known from theory and synoptic experience that the larger the scale of an atmospheric system, the longer is its timescale and the more predictable it normally is (see Fig4.1-2).

Fig4.1-2:  A schematic illustration of the relationship between atmospheric scale and timescale. The typical predictability is currently approximately twice the timescale, but might ultimately be three times the timescale.


The typical predictability is currently approximately twice the timescale, but might ultimately be three times the timescale.  Small baroclinic systems or fronts are currently well forecast to around Day2, cyclonic systems to around Day4 and the long planetary waves defining weather regimes to around Day8.  As models improve over time these limits are expected to advance further ahead of the data time.  Features that are coupled to the orography (e.g. lee-troughs), or to the underlying surface (e.g. heat lows), are rather less consistently well forecast.  The predictable scales also show the largest consistency from one run to the next.  Fig4.1-3 shows 1000hPa forecasts from six sequential runs of HRES (identical to CTRL) verifying at the same time. 


Fig4.1-3: A sequence of Mean Sea Level Pressure forecast charts ranging from T+156 to T+96, all verify at 00UTC 24 October 2022.  The forecast details differ between the forecasts but large-scale systems (a low near Ireland, a high over central Europe, a trough towards the southern Baltic) are common features.  The T+156 predicted gales over southern and northwest France.  It would have been unwise to make such a detailed interpretation of the forecast, considering the typical skill at that range.  Only a statement of windy, unsettled and cyclonic conditions would have been justified.  Such a cautious interpretation would have avoided any embarrassing forecast “jump”, when the subsequent T+144 and T+132 runs showed a weaker circulation.  The same cautious approach would have minimized the forecast “jump” with the arrival of the T+108 forecast.


 A synoptic example of combining EM and probabilities

However, spectral filtering does not take into account how the predictability varies due its flow dependency.  A small-scale feature near Portugal might be less predictable than an equally sized feature over Finland.

Over-interpreting the ensemble forecast or underestimation of the risk of extreme weather events should be avoided.  This can be helped if ensemble forecasts are presented together with a measure of the ensemble spread or event probabilities.   As an example, gales are put into a synoptic context if ensemble forecasts of MSLP (or 1000hPa) are presented together with gale probabilities (Fig4.1-4).

Fig4.1-4: A sequence of Mean Sea Level Pressure forecast charts overlaid by the probabilities of wind speeds >10 m s-1 ranging from T+156 to T+96, all verify at 00UTC 24 October 2022.  Probabilities are coloured according to the scale. Compare with Fig4.1-3.

The ensemble assesses the forecasts in a consistent and optimal way as its flow dependency serves as a superior dynamic filter.   It gives the probability of an outcome (in this case strength of wind) rather than relying on an individual solution.

 The T+12 ensemble forecast is used here as an analysis proxy for the verification of the above forecasts (see Fig4.1-5).

 

 Fig4.1-5: 1000 hPa ensemble mean DT 19 August 12 UTC T+12 VD 20 August 00 UTC may serve as a proxy analysis for verification because of the small forecast range and the fact that the EM, thanks to the initial anti-symmetric nature of the perturbations, is almost identical to the CTRL. The probabilities essentially show where the verifying wind speed was > 10 m/s.

It can be seen from the above that some of the HRES (identical to CTRL) forecasts in Fig4.1-3 (T+96, T+108 and perhaps T+144) were quite good with respect to strong winds over Britain and Ireland but at the time the ensemble indicated that gale force winds were not certain. 

Model drift

In order to estimate and compensate for any model drift the model output is compared with the corresponding model climates (M-climate for medium range, ER-M-climate for extended range, S-M-climate for seasonal forecasting) for the current forecast date.  This is derived using the same model construction as the ensemble from a number of perturbed forecasts based on calendar dates surrounding the date of the current ensemble run using historical data from several years.  Systematic errors are then corrected during post-processing after the forecast is run.