Page History

Relative differences in error with lead-time among HRES, CTRL and individual ensemble members.

It is useful to note the relative differences in error with lead-time among HRES, ensemble CTRL and a sample individual ensemble member of ENS.  It is also useful to understand how the errors grow and saturate at different spatial scales.

The plots below are from the diagnostic monitoring of the forecast system, which is done on a season-to-season basis, and so are based on 3 months of data. The dots indicate aspects which are statistically significant based on the data used.

Image Removed

Fig6.4.9: Plot showing for HRES (red); CTRL (blue):

• squared forecast error ((Forecast-Analysis)²; full lines),
• "activity" in the forecast (2x(Forecast-Climatology)²; dotted lines),
• "activity" in the analysis (2x(Analysis-Climatology)²; dashed lines).

These values are calculated for 500hPa heights over the Northern Hemisphere during the period Dec 2016-Feb 2017 and plotted against forecast lead-time.  Spatial scales greater than about 150km (total wavenumbers 0-63) are considered here.  All values are normalised by the same amount so that the maximum value plotted is 1.  The dots on the error curves indicate that the difference is statistically significant.

Fig6.4.9 shows that HRES errors (red full line) are marginally smaller than CTRL errors (blue full line) when concentrating on scales greater than about 150km (total wavenumbers up to 63) and medium-range lead-times. 

Image Removed

   The diagrams below were derived from cycles before CY48r1 and refer to resolutions of 18km.

Image Added

Fig6.2.4-1Fig6.4.10: Plot showing for CTRL (red); a single ensemble member (blue):.  Waves 0-63.

• squared forecast error ((Forecast-Analysis)²; full lines),
• "activity" in the forecast (2x(Forecast-Climatology)²; dotted lines),
• "activity" in the analysis (2x(Analysis-Climatology)²; dashed lines).  
  

These values are calculated for 500hPa heights over the Northern Hemisphere during the period Dec 2016-Feb 2017 and plotted against forecast lead-time. Spatial scales greater than about 150km (total wavenumbers 0-63) are considered here. All values are normalised by the same amount so that the maximum value plotted is 1. The dots on the error curves indicate that the difference is statistically significant.

Fig6.2.4.10 -2 shows that there is a bigger difference between CTRL ensemble control (red full line) and one of the perturbed ensemble members (blue full line).  By comparing the lead-times at which CTRL ensemble control and the single member reach the same error, we see that the CTRL ensemble control is just short of a day better than a single ensemble member. It should be remembered here that the ENS is designed to explore the less likely outcomes and, taken as a whole, it generally contains more information than the CTRL ensemble control (or the HRES).  On the other hand, at short lead-times, the ensemble control (or the HRES) can contain additional information at smaller spatial scales.

Fig6.2.4.11-2: Plot showing for CTRL (red); a single ensemble member(blue) in.  Waves 0-3 and 4-14.:

• squared forecast error ((Forecast-Analysis)²; full lines),
• "activity" in the forecast (2x(Forecast-Climatology)²; dotted lines),
• "activity" in the analysis (2x(Analysis-Climatology)²; dashed lines).  

These values are calculated for 500hPa heights over the Northern Hemisphere during the period Dec 2016-Feb 2017 and plotted against forecast lead-time. Planetary spatial scales greater than about 2500km (zonal wavenumbers 0-3) are shown with thick lines, synoptic spatial scales 650-2500km, (zonal wavenumbers 4-14) are shown with thin lines. The dots on the error curves indicate that the difference is statistically significant.

Fig6.2.4.11 -2 shows the contributions to the forecast errors from planetary-wave scales (scales greater than about 2500km, with zonal wavenumbers wave numbers 0-3; thick full lines) and synoptic scales (scales between 650-2500km, with zonal wavenumbers wave numbers 4-14; thin full lines).  The CNTL ensemble control has a similar lead-time advantage (just less than a day) over the single member for both scales. Up to about Day 7, the contributions from both scales to the total error are quite similar (the thick and thin full lines are on top of each other). Beyond Day 7, the contribution from the synoptic scales begins to saturate (i.e. have little skill) as it approaches the atmospheric activity at these scales (thin dotted and dashed lines). On the other hand, the planetary scales continue to include useful information beyond Day 7, and their error continues to increase towards the (higher) level of planetary scale activity (thick dotted and dashed lines).