Note: HRES and Ensemble Control Forecast (ex-HRES) are scientifically, structurally and computationally identical.  With effect from Cy49r1, Ensemble Control Forecast (ex-HRES) output is equivalent to HRES output shown in the diagrams.   At the time of the diagrams, HRES had resolution of 9km and ensemble members had a resolution of 18km.

The Equitable Threat Score (ETS) measures the skill of a forecast relative to chance and is often used in the verification of precipitation forecasts.  It is computed for each station over the given period, and then averaged over an area.  

When interpreting the ETS for precipitation note that: 

  • the scale mismatch between the point measurement (rain gauge) and the grid box value from the NWP model reduces the score.
  • there are pronounced geographical variations (e.g. higher ETS in areas where orographic precipitation is dominant).
  • the seasonal cycle is quite pronounced, with values dropping significantly in summer when convection is active.

 

The diagram below was derived from cycles before Cy48r1 when HRES had 9Km resolution, the medium range ensemble had 18Km resolution. 

 Fig6.2.3-1:  The Equitable Threat Score (ETS) measures the skill of a forecast relative to chance.  Here it considers observations and forecasts of ≥1mm measured in a day (in effect a rain/no rain verification) at each station during 2016 and averaged over the extra-tropical Northern Hemisphere.  HRES in Red, ensemble control (CTRL) in purple, Ensemble mean (EM) in green, An individual ensemble member (PF) in cyan.  Note the Equitable Threat Score for ensemble control (CTRL) and HRES are very similar, but almost a day better than any individual member (e.g. the skill of ensemble control (CTRL) at Day7 is as good as that of an individual perturbed forecast PF at Day6, or the skill HRES at Day3 is as good as an individual perturbed forecast PF at day 2).  Note at Day2 the HRES has a higher skill than the ensemble control (CTRL) but by Day10 both HRES and ensemble control (CTRL) have similar (low) skill.   In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.



(FUG Associated with Cy49r1)