Section 6.2.4 Relative error differences between models

Relative differences in error with lead-time among HRES, CTRL and individual ensemble members.

It is useful to note the relative differences in error with lead-time among ensemble CTRL and a sample individual ensemble member of ENS.  It is also useful to understand how the errors grow and saturate at different spatial scales.    The diagrams below were derived from cycles before CY48r1 and refer to resolutions of 18km.

Fig624.A: Plot showing for CTRL (red); a single ensemble member (blue).  Waves 0-63.

• squared forecast error ((Forecast-Analysis)²; full lines),
• "activity" in the forecast (2x(Forecast-Climatology)²; dotted lines),
• "activity" in the analysis (2x(Analysis-Climatology)²; dashed lines).  
  

These values are calculated for 500hPa heights over the Northern Hemisphere during the period Dec 2016-Feb 2017 and plotted against forecast lead-time. Spatial scales greater than about 150km (total wavenumbers 0-63) are considered here. All values are normalised by the same amount so that the maximum value plotted is 1. The dots on the error curves indicate that the difference is statistically significant.

Fig624.A shows that there is a bigger difference between ensemble control (red full line) and one of the perturbed ensemble members (blue full line).  By comparing the lead-times at which ensemble control and the single member reach the same error, we see that the ensemble control is just short of a day better than a single ensemble member. It should be remembered here that the ENS is designed to explore the less likely outcomes and, taken as a whole, it generally contains more information than the ensemble control (or the HRES).  On the other hand, at short lead-times, the ensemble control (or the HRES) can contain additional information at smaller spatial scales.

Fig624.B: Plot showing for CTRL (red); a single ensemble member(blue).  Waves 0-3 and 4-14.:

• squared forecast error ((Forecast-Analysis)²; full lines),
• "activity" in the forecast (2x(Forecast-Climatology)²; dotted lines),
• "activity" in the analysis (2x(Analysis-Climatology)²; dashed lines).  

These values are calculated for 500hPa heights over the Northern Hemisphere during the period Dec 2016-Feb 2017 and plotted against forecast lead-time. Planetary spatial scales greater than about 2500km (zonal wavenumbers 0-3) are shown with thick lines, synoptic spatial scales 650-2500km, (zonal wavenumbers 4-14) are shown with thin lines. The dots on the error curves indicate that the difference is statistically significant.

Fig624.B shows the contributions to the forecast errors from planetary-wave scales (scales greater than about 2500km, with zonal wave numbers 0-3; thick full lines) and synoptic scales (scales between 650-2500km, with zonal wave numbers 4-14; thin full lines).  The ensemble control has a similar lead-time advantage (just less than a day) over the single member for both scales. Up to about Day 7, the contributions from both scales to the total error are quite similar (the thick and thin full lines are on top of each other). Beyond Day 7, the contribution from the synoptic scales begins to saturate (i.e. have little skill) as it approaches the atmospheric activity at these scales (thin dotted and dashed lines). On the other hand, the planetary scales continue to include useful information beyond Day 7, and their error continues to increase towards the (higher) level of planetary scale activity (thick dotted and dashed lines.