Page History

...

Just because the most recent forecast is, on average, better than the previous one, it does not mean that it is always better.  A more recent forecast can be worse than a previous one, and often with increasing forecast range it becomes increasingly likely that the 12 or 24 hours older forecast may be the better one.  If the most recent NWP model output differs significantly from previous results the forecaster can use techniques outlined below in order to avoid sudden changes in forecasts being given to the customer.  It can be worthwhile trying to assess the cause of the difference, but in general each ensemble solution should be viewed as one possible solution that is a member of a greater ensemble of the latest and recent solutions.

Ensemble of Data Assimilations (EDA), Ocean coupling from Day0, and future enhancements to stochastic physics and land surface perturbations are designed to improve the quality of the ensemble and should continue to reduce, though not eradicate, jumpiness.

...

Fig7.2.2: An example of a forecast exhibiting jumpiness in the form of a major flip-flop.  On the diagram the y-axis is for forecast 2m temperatures, the x-axis shows the data time of the ensemble forecast.  The plotted values are for the forecast temperature at Paris verifying at 00Z 8 Dec 2016.  Forecast ensemble results are shown by box and whisker plots (described in Meteogram section), forecast ensemble mean values shown by black dots (red dots show values from the HRES).  Initially Day15 to Day11 forecasts were around 5°C or 6°C although with a broad range of up to ±8 to 10°C.   From 12UTC 27 Nov (Day10½) the forecast temperatures jumped to much colder values round -2°C with a relatively small spread of ±3 or4°C.  From 12UTC 30 Nov (Day7½) the forecast temperature rose suddenly back to around +6°C with a broader spread of ±8 to 10°C.  From 12UTC 3 Dec (Day4½) the forecast temperatures reverted to around +4°C with range of ±2 or 3C.   It should be remembered in general each ensemble solution should be viewed as one possible solution that is a member of a greater ensemble of the latest and recent solutions, although the later solutions do have the benefit of the most up to date data.

...

Fig7.2.3A: An example of a forecast showing a major flip with a major difference in the forecast depth and extension of the upper trough over Eastern Europe.  The charts show 500hPa forecasts at 9km resolution VT 00UTC 25 Dec 2012 - upper chart: T+144 DT 00UTC 19 Dec 2012 with cold air across forecast Greece and SE Europe, lower chart: T+120 DT 00UTC 20 Dec 2012 with warm airmass over Greece and SE Europe.

Fig7.2.3B: Standard ECMWF Mean and Spread charts for 850hPa temperature,verifying at 00UTC 25 Dec 2012, T+120 from ensemble control (CTRL) data time 00UTC 20 Dec 2012 (same case as in the lower panel of Fig7.2.3A).

...

Persson and Strauss (1995), Zsótér et al. (2009) found:

...

...

• adjust a forecast value (e.g. temperature, rainfall, etc) slightly lower or higher to follow the latest indications (e.g. warmer/cooler, wetter/drier, etc), but nevertheless to remain within the range of ensemble solutions from the latest and previous runs.  Reducing the change suggested by a noteworthy jump in the forecast can be the most appropriate course of action - but it does run the risk that the forecast from the next run will be even further away from the earlier solutions (i.e. the forecaster could be trying catch up with the NWP model forecasts and this illustrates one of the ways in which accuracy will be reduced).  On the other hand, it should be remembered that to follow a trend is also unreliable ~50% of the time.

• check whether the ensemble mean and probabilities are fairly consistent with previous runs.   If not, consider creating a lagged ensemble of the last two or three ensemble forecasts to give two or three times the number of members.  This will smooth out sudden changes in evolution while preserving the breadth of possible forecast extremes and probability information from the latest run.  A grand ensemble of ECMWF forecast results may be considered to compare latest forecast results with those of other state-of-the-art NWP models. 
  

• follow the ensemble mean rather than the ensemble control.  This can be more informative, especially at longer lead-times (say ≥ ~ 4 days).   However, note that strong gradients are always weakened in the ensemble mean and fine scale features (e.g. sting jets) will not be visible.
• inspect the Cumulative Density Function (CDF) of ensemble forecasts.  This can give a useful indication on the ensemble forecast values during the jumpiness.  At longer lead-times forecast CDFs may be similar to the M-climate.  But, with time, CDF between successive runs should show less lateral variation and tend to become steeper implying higher confidence.
  
  

Fig7.2.7:An example of Cumulative Density Function (CDFs ) produced by a sequence of ensemble forecasts for precipitation at Zaga in Slovenia verifying for the 24hr 00UTC 27 to 00UTC 28 April 2017.  All show a very high extreme forecast index (EFI).  Note the four earlier CDFs (blues) showed a moderate slope indicating a spread of forecast precipitation intensities, and then jumped to a steeper slopes (purple and red) with lessening of spread of precipitation intensities.  Here the forecast showed a steady trend towards heavier precipitation with a jump to very heavy precipitation.  A forecaster would have been unwise at the time of the T+60 to 84hr forecast (rightmost dashed blue line) to think that this significantly wetter forecast overall was too much of  jump from the trend to be believed.

Special considerations - Jumpiness at short lead-times

...