Page History

...

Perturbations, positive and negative, spread the ensemble forecasts either side of the ensemble control (CTRL) early onin the forecast, so and any jumps in CTRL (and HRES) the ensemble control are likely be shown by ENS the ensemble also.  At very short lead-times, before perturbations have had time to amplify, the ensemble mean (EM) will be very similar to the CTRLensemble control.   Later in the forecast non-linearity becomes more important, and so the ENS ensemble members are less similar to the CTRL (and HRES), making ensemble control.  Thus the ensemble mean (EM) forecast to beis, on average, a less jumpy and a more reliable forecast than the CTRLensemble control.

Just because the most recent forecast is, on average, better than the previous one, it does not mean that it is always better.  A more recent forecast can be worse than a previous one, and often with increasing forecast range it becomes increasingly likely that the 12 or 24 hours older forecast is may be the better one.  If the most recent NWP model output differs significantly from previous results the forecaster can use techniques outlined below in order to avoid sudden changes in forecasts being given to the customer.  It can be worthwhile trying to assess the cause of the difference, but in general each HRES and ENS solution should be viewed as one possible solution that is a member of a greater ensemble of the latest and recent solutions.

...

 Fig7.2.1:  A representation of forecast temperatures at a certain location for a certain date produced by a series of forecast runs.  A jump may be considered as greater than some threshold δ.  Flip-flop may be considered as a sequence of results that alternately are higher or lower than its predecessor.  A trend may be considered as a sequence of results that rise or descend, uniformly or not.

In a sequence of 3 three forecasts, there are only two ways the final forecast in that sequence can behave relative to the others - it can represent a trend, or it can represent a flip-flop (assuming that we discount the possibility of sequential forecasts being identical).  Both occur about 50% of the time, so a forecaster should not be surprised when one of these occurs, nor should any special action be taken according to which it is.

...

Sometimes forecasts show significant and repetitive changes in predictions for a given location.  Often this is associated with the precise positioning of a trough or ridge in the vicinity of the location of interest (e.g. if in the Northern Hemisphere the axis lies to the east, then a northerly airflow brings colder temperatures and an associated type of weather; if to the west, then a southerly airflow brings warmer temperatures and a different weather type).  

...

Fig7.2.2: An example of a forecast exhibiting jumpiness in the form of a major flip-flop.  On the diagram the y-axis is for forecast 2m temperatures, the x-axis shows the data time of the ENS or HRES forecastensemble forecast.  The plotted values are for the forecast temperature at Paris verifying at 00Z 8 Dec 2016.  Forecast ENS results are shown by box and whisker plots (described in Meteogram section), forecast ensemble mean (EM) values shown by black dots , forecast HRES values by (red dots .show values from the now obsolete HRES ).  Initially Day15 to Day11 forecasts were around 5°C or 6°C although with a broad range of up to ±8 to 10°C.   From 12UTC 27 Nov (Day10½) the forecast temperatures jumped to much colder values round -2°C with a relatively small spread of ±3 or4°C.  From 12UTC 30 Nov (Day7½) the forecast temperature rose suddenly back to around +6°C with a broader spread of ±8 to 10°C.  From 12UTC 3 Dec (Day4½) the forecast temperatures reverted to around +4°C with range of ±2 or 3C.  The HRES forecasts showed even greater volatility, notably around 1 Dec (Day8).  Around 30 Nov, it would have been unwise to follow the HRES closely as the forecast values are very much on the wings of ENS probability but there there would have been a case to raise forecast temperatures to lie within the higher probability values of the ENS (as shown by the box of the box and whisker plot).  range of ±2 or 3C.   It should be remembered in general each HRES and ENS solution should be viewed as one possible solution that is a member of a greater ensemble of the latest and recent solutions, although the later solutions do have the benefit of the most up to date data.

...

Fig7.2.3A: An example of a forecast exhibiting a major flip with a major difference in the forecast depth and extension of the upper trough over Eastern Europe.  The charts show HRES 500hPa forecasts verifying at 00UTC 25 Dec 2012 - upper chart: T+144 data time 00UTC 19 Dec 2012 with cold air across forecast Greece and SE Europe, lower chart: T+120 data time 00UTC 20 Dec 2012 with warm airmass over Greece and SE Europe.

...

Left panel: Ensemble Mean and Normalized Standard Deviation chart shows the ensemble mean (EM) temperature as isotherms, whilst shading shows the normalised spread.  This normalised spread denotes how spread in the latest ensemble at a location compares with the ensemble spread at that location over recent (the last 30 days of) forecasts.  This chart, therefore, highlights uncertainty (green – relatively low, purple( relatively high). So at Belgrade (blue circle) there is relatively high spread (uncertainty) and consequently one has less confidence in the forecast than might otherwise be expected for T+120 in that area. Conversely the green area to the south of Ireland denotes less spread than seen in recent ENS forecasts.

Right panel: The deterministic (HRES) forecast and the actual ensemble Standard Deviation.  This shows that in absolute terms (as well as relative terms) there is a wide spread of ensemble solutions over eastern Europe (standard deviation between 4.5°C and 8.0°C).

...

Viewing Figure 7.2.3B alongside 7.2.3A, one can find evidence for why the large jump in the deterministic forecast in the vicinity of Belgrade should not have come as such a surprise. Uncertainty in absolute and relative terms was still very high after the jump. This is even though the ensemble mean temperature value after the jump mirrored the HRES (deterministic) value quite well (contours on Fig7.2.3B). So  So in broad terms the forecaster would be justified in following the jump, but at the same time should assign a large error bar to any issued forecasts.

...

It is important to recognise that intuitive interpretation of jumpiness – flip-flopping and trends– is not reliable although appealing.   And in particular, one should not assume that a forecast is more reliable if it has not changed substantially from the previous run.  A similar forecast should be regarded as fortuitous rather than definitive.  Rather, inspection of the ensemble forecasts (ENS) will give an insight into the probability of the outcome.

The ENS and HRES forecasts must differ from one run to the next In order for the forecast output to gradually improve.  There must always be variations in evolution due to the incorporation of data in the analysis scheme and through field interactions during that evolution.  Ideally in ENS these “forecast jumps” should be fairly regular with no “flip-flopping”.   

Uncertainty increases with forecast range; the jumpiness increases in HRES any individual ensemble member though, in relative terms, it decreases in the EMensemble mean.  In summary:     

• at very short range, the EM ensemble mean is almost identical to the CTRL and HRESensemble control, and jumpiness is about equally low for both,
• at short lead-times, large jumpiness does not mean there is a large error in the latest forecast,  
• at longer lead-times larger jumpiness implies greater error, but the correlation is no more than 0·5.

...

• The proportion of previous forecasts that are "better" than the latest ones increases with lead-time:
  
  • at short lead-times a small but significant proportion appear better (~15% at Day2),
  • at longer lead-times a larger a larger proportion appear better (~40% at Day6).  (Fig7.2.4).
• There is only a very small correlation between forecast jumpiness and the quality of the latest forecast (Fig7.2.5).
  
• The EM Beyond about Day3 the ensemble mean, by using results from all ensemble members, provides more consistent forecasts than the ensemble control (CTRL) beyond about Day3.  This benefit gradually increases with forecast range.  
• The frequency of a flip (single jump) is very similar for both the EM and the CTRLensemble mean and ensemble control.
• The frequency of flip-flopping occurs clearly less frequently in the EM than for the CTRL ensemble mean than in the ensemble control.

Persson and Strauss (1995), Zsótér et al. (2009) found:

• the connection between forecast inconsistency (flip-flopping etc) and forecast error is weak,
• the average error of the EM ensemble mean relates quite strongly to the absolute spread in the ensemble.  
• on average, larger spread implies larger errors (this does not apply to the ensemble median , CTRL or HRESensemble control, even if they happen to lie mid-range within the ensemble).

...

• adjust a forecast value (e.g. temperature, rainfall, etc) slightly lower or higher to follow the latest indications (e.g. warmer/cooler, wetter/drier, etc), but nevertheless to remain within the range of ENS ensemble solutions from the latest and previous runs.  Reducing the change suggested by a noteworthy jump in the forecast can be the most appropriate course of action - but it does run the risk that the forecast from the next run will be even further away from the earlier solutions (i.e. the forecaster could be trying catch up with the NWP model forecasts and this illustrates one of the ways in which accuracy will be reduced).  On the other hand, it should be remembered that to follow a trend is also unreliable ~50% of the time.

• check whether the EM ensemble mean and probabilities are fairly consistent with previous runs.   If not, consider creating a lagged ensemble of the last two or three ensemble forecasts to give two or three times the number of members.  This will smooth out sudden changes in evolution while preserving the breadth of possible forecast extremes and probability information from the latest run.  A grand ensemble of ECMWF forecast results may be considered to compare latest forecast results with those of other state-of-the-art NWP models. 
  

• follow the ensemble mean (EM), rather than the ensemble control (CTRL) or HRES.  This can be more informative, especially at longer lead-times (say ≥ ~ 4 days).   However, note that strong gradients are always weakened in the ensemble mean and fine scale features (e.g. sting jets) will not be visible.
• inspect the Cumulative Density Function (CDF) of ensemble forecasts.  This can give a useful indication on the ENS forecast values during the jumpiness.  At longer lead-times forecast CDFs may be similar to the M-climate.  But, with time, CDF between successive runs should show less lateral variation and tend to become steeper implying higher confidence.
  
  

...

In ‘finely balanced’ situations (those with dynamical sensitivity), the ensemble spread can be quite high even at quite short lead-times (about one or two days); slight differences and jumpiness in ENS, HRES and CTRL among ensemble members or control can all have a large impact on the NWP model evolution (e.g. precise phasing of upper and lower levels needed for explosive cyclogenesis; high precipitation intensities can turn rain into (surprise) snow due to cooling through melting).  Severe weather situations are often associated with these sorts of uncertainties and a probabilistic approach rather than definitive forecast is generally more effective and useful.

...