The Anomaly Correlation Coefficient (ACC) is one of the most widely used measures in the verification of spatial fields. It is the spatial correlation between a forecast anomaly from climatology and a verifying analysis anomaly from climatology. ACC represents a measure of how well the forecast anomalies have represented the observed anomalies.
ACC values lie between +1 and -1. Where ACC values:
- approach +1 there is good agreement and the forecast anomaly has had value.
- lie around 0 there is poor agreement and the forecast has had no value.
- approach –1 the agreement is in anti-phase and the forecast has been very unreliable.
Where the ACC value falls below 0.6 it is considered that the positioning of synoptic scale features ceases to have value for forecasting purposes.
At ECMWF the ACC scores represent the spatial correlation between:
- the anomaly of theSEAS5 forecast measured against a model climatology based on the ERA-interim re-analysis (based on the period 1993-2016) and
- the anomaly of the verifying observations measured against a model climate (S-M-climate) based on the ERA-interim re-analysis (based on the period 1981-2016).
In the analysis below (Fig6.4.1 and Fig6.4.2), forecasts by all IFS models progressively correlate less well with observed values as forecast lead-time increases. The unperturbed HRES and the ensemble control (CTRL) perform similarly, and better than any individual perturbed member of ENS (PF in the diagrams). The ENS mean smooths out many smaller scale features and performs best against the verifying analysis. This supports preferential use of the ENS mean in practical forecasting. Typically ACC falls to 0.6 around day8 or day9 for HRES and CTRL, and around day10 for the ensemble mean (EM).
Fig6.4.1: Anomaly Correlation Coefficients for 850hPa Temperatures. HRES in Red, Ensemble control (CTRL) in Purple, Ensemble mean (EM) in Green, An individual ensemble member (PF) in Cyan. Note the ACC for CTRL and HRES are very similar, but the EM clearly out-performs them and is almost 2½ days better than any individual ENS member (e.g. ACC of PF at Day7 is still being attained by EM at Day9½) .
Fig6.4.2: Anomaly Correlation Coefficients for 500hPa Geopotential. HRES in Red, Ensemble control (CTRL) in Purple, Ensemble mean (EM) in Green, An individual ensemble member (PF) in Cyan. Note the ACC for CTRL and HRES are very similar, but the EM clearly out-performs them and is almost 2 days better than any individual ENS member (e,g, ACC of PF at Day7 is still being attained by EM at Day9) .
The diagrams above show how the Anomaly Correlation Coefficients vary with forecast lead-time. The ACC scores have steadily improved over the years at all lead-times, reflecting the improvement of input data, analysis techniques and IFS model formulation (See Fig6.4.3).
Fig6.4.3: Time series of the annual running mean of anomaly correlations of HRES 500 hPa height forecasts evaluated against the operational analyses for the period 2000 until 2017. Values are running 12month average scores. Forecast lead-times in days ahead - 3(blue), 5(red), 7(green) and 10(yellow) - are shown for scores averaged over the northern extra-tropics (bold lines) and southern extra-tropics (thin lines). The shading shows differences in scores between the two hemispheres at the forecast ranges indicated. Currently 3-day, 5-day, 7-day, 10-day forecasts have attained approximately 98%, 92%, 78%, 50% anomaly correlation (ACC) respectively.