Relative weights for HRES and ENS
Relative weights that can be applied to HRES and ENS
The Anomaly Correlation Coefficient shown in the previous sections highlighted differences between the various forecast components when averaged over a whole hemisphere. While this is useful information, a forecaster would clearly benefit from knowing about skill differences for smaller regions (or even point locations) and the weights to attach to each forecast component. For example, the HRES might be expected to capture the detail of small-scale features better at shorter lead-times while the ENS, with its many members, might be expected to better describe the diversity (or distribution) of possible outcomes at longer lead-times. Intuitively, more reliance might be placed on the HRES at short ranges, and more confidence might be placed in the ENS at mid- and longer ranges. However, i and the t is important to be able to quantify these ideas so that due weight can be given to each in a more structured way (e.g. for specific sites, lead-times and predicted variables). Fig6.4.5 shows the weights that are obtained, for 2m temperatures at Vienna and Skopje, when the HRES, CTRL and ENS (or just the CTRL and ENS) are combined to produce a normally-distributed forecast with the most probabilistic skill. At Day 1, the HRES is given more weight than the ENS. At Day 10, the ENS is given much more weight than the HRES. In general, the CTRL has little weight throughout the forecast range.
Fig6.4.5: Optimal relative weights of ENS, CTRL and HRES forecasts at various lead-times. The left hand side of each panel (labelled "Full Model") shows the weights assigned to HRES, CTRL, and ENS respectively by Ensemble Model Output Statistics (EMOS) optimised for the Continuous Ranked Probability Score (CRPS) of T2m at (a) Vienna Hohe Warte and (b) Skopje. The right hand side of each panel (labelled "Without HRES") shows the weights for CTRL and ENS when HRES is not included in the calibration. More reliance can be placed on the HRES at short periods, but more confidence may be placed in the ENS at mid- and longer periods. CTRL used by itself generally has a low utility and the ensemble members prove much better for guidance but CTRL approaches the effectiveness of HRES at longer lead-times. In the histograms for Skopje, at Day1 the HRES should have about 67% weight while ENS only about 33% weight in any forecaster assessment of likely conditions, while by Day10 HRES should have only 10% weight and ENS 90% weight.
An alternative approach, which does not require a distribution assumption is to simply insert the HRES into the ensemble, as shown schematically in Fig6.4.6, and to estimate how many ensemble members it is equivalent to (in Fig6.4.6 it is shown as equivalent to 3 members of a 10-member ensemble). In Fig6.4.7, the weight of the HRES was calculated to optimise the Brier Skill Score of point precipitation over Europe for the three different precipitation thresholds. Broadly speaking the HRES is equivalent to 14-18 members at a lead-time of 1 day and this then falls to just 1-3 members by Day10. Evidently the HRES is providing useful detailed information on scales that are not resolved by the ENS at Day1, but this detail becomes superfluous at longer lead-times, when the uncertainty at synoptic scales is much larger, and the HRES may be considered almost as just part of the ensemble. These average weightings will vary from location to location, with local orographic effects, and over the annual cycle but, even so, Fig6.4.8 shows that using these average weights leads to statistically significant improvements in skill at all lead-times.
Fig6.4.6: A hypothetical example of computation for a particular site, using weights, where there is one HRES and a 10 member ensemble. There are 6 ENS members showing >1mm rainfall in a day and the weight to be given to HRES is 3 ENS members. So the probability of 1mm rainfall in a day is (ENS equivalence of HRES + ENS rainfall >1mm) / (ENS equivalence of HRES + total ENS) = (3+6) / (3+10) = 9/13.
Fig6.4.7: The optimal weight to give the HRES for the case of probability of precipitation > 1mm/day (red), > 5mm/day (green), >10mm/day (blue). At short lead-times, HRES is very valuable (e.g. for >1mm/day it is worth about 18 EPS members, and for >10mm/day it is worth about 14 members). At longer lead-times the weight to be given to HRES becomes more similar to the weight to be given to ENS members (e.g. for >1mm/day it is worth about 3 ENS members, and for >10mm/day it is worth about 1 member). Based on a cross-validated optimisation of skill when verified against European SYNOP observations of precipitation for the years 2001-2005.
Fig6.4.8: Plot of Brier Skill Score of a)the combined system of HRES plus ENS (orange) and b)ENS alone (black) for probability of precipitation (Pp). If weighting is applied to HRES appropriate to each lead-time, the outcome is significantly more skilful (on average) at all lead-times and all thresholds than by using ENS alone. Results are cross-validated so there is no artificial inflation of skill. For Pp >1mm/day, the additional skill of HRES+ENS over ENS alone is about 10 hours at Day2. Verified against European SYNOP observations of precipitation for the years 2001-2005.
The weighting tool in ecCharts (in the "add layers" filter box type "weight") allows the user to investigate the impact of higher resolution forecast results (possibly with a view to automation in the future).
Additional Sources of Information
(Note: In older material there may be references to issues that have subsequently been addressed)
- Read about calibration of ECMWF forecasts.