Page History

The Anomaly Correlation Coefficient (ACC) is one of the most widely used measures in the verification of spatial fields.   It is the spatial correlation between a forecast anomaly relative to climatology, and a verifying analysis anomaly relative to climatology.  ACC represents is a measure of how well the forecast anomalies have represented the observed anomalies and .  It shows how well the predicted values from a forecast model "fit" with the real-life data.  When ACC is calculated for a sequence of forecasts, say at  ACC for a series of forecast lead-times , it is a measure of how well trends in the predicted anomalies follow trends in actual anomalies.

...

At ECMWF the anomaly correlation coefficient (ACC) scores represent the spatial spacial correlation between:

• the mean of the anomalies of a forecast product relative to reference model climate and
• the mean of the anomalies of observations or reanalysis relative to the same reference model climate.

...

• For medium range forecasts the anomaly correlation coefficient is evaluated between:
  
  • the mean of the anomaly of the forecast product relative to the medium range model climate (M-climate) and
  • the mean of the anomaly of the verifying HRES or CTRL analysis relative to the medium range model climate (M-climate).
• For extended range products the correlation is evaluated between:
  • the mean of the anomaly of the forecast product measured relative to the extended range model climate (ER-M-climate) and
  • the mean of the anomaly of the verifying the CTRL analysis relative to the extended range model climate (ER-M-climate).
• For seasonal products the correlation is evaluated between:
  • the mean of the anomaly of the forecast product relative to the a model climatology based on the ERA-interim re-analysis (based on the period 1993-2016) and
  • the mean of the anomaly of the verifying observations or reanalysis relative to the seasonal model climate (S-M-Climate). 

The The medium range model climatemodel climate (M-climate) and  and extended range model climate (ER-M-climate) are  are based on reon re-forecasts spanning the last 20 years, which used the ERA5 reanalysis for their initialisation

The seasonal model climate (S-M-climate) is based on re-forecasts spanning the last 20 years, which used the ERA-interim reanalysisre-analysis for their initialisation

 In the analysis below (Fig6.42.2-1 and Fig6.42.2-2), forecasts by all IFS models progressively correlate less well with observed values as forecast lead-time increases.   The unperturbed HRES and the ensemble  In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.  The unperturbed ensemble control (CTRL) perform similarly, and performs better than any individual perturbed member of ENS ensemble (PF in the diagrams).  The ENS ensemble mean smooths out many smaller scale features and performs best against the verifying analysis.  This supports preferential use of the ENS ensemble mean in practical forecasting.  Typically  Typically ACC falls to 0.6 at around day8 or day9 for HRES and ensemble control (CTRL), and at around day10 for the ensemble mean (EM).

 Fig6.42.2-1: Anomaly Correlation Coefficients for 850hPa Temperatures. HRES in Red, Ensemble control (CTRL) in Purple, Ensemble mean (EM) in Green, An individual ensemble member (PF) in Cyan.  Note the ACC for CTRL ensemble control and HRES are very similar, but the EM ensemble mean clearly out-performs them and is almost 2½ days better than any individual ENS ensemble member (e.g. ACC of a perturbed member (PF) at Day7 is still being attained by EM ensemble mean at Day9½).  In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.  

 Fig6.42.2-2: Anomaly Correlation Coefficients for 500hPa Geopotential.  HRES in Red, Ensemble control (CTRL) in Purple, Ensemble mean (EM) in Green, An individual ensemble member (PF) in Cyan.    Note the The ACC for CTRL ensemble control and HRES are very similar, but the EM .  However, ensemble mean clearly out-performs them and is almost 2 days better than any individual ENS member.  ACC of a perturbed member (e,g, ACC of PF) at Day7 is still being attained by EM ensemble mean at Day9).  In these graphs HRES has 9Km resolution, the medium range ensemble has 18Km resolution.  

The diagrams above show how the Anomaly Correlation Coefficients vary with forecast lead-time.  The ACC scores have steadily improved over the years at all lead-times, reflecting .  This reflects the improvement of input data, analysis techniques and IFS model formulation (See Fig6.42.2-3).

 Fig6.42.2-3:  Time series of the annual running mean of anomaly correlations of HRES 500 hPa height forecasts evaluated against the operational analyses for the period 2000 until 20172022.  Values are running 12month average scores.  Forecast lead-times in days ahead - 3(blue), 5(red), 7(green) and 10(yellow) - are shown for scores averaged over the northern extra-tropics (bold lines) and southern extra-tropics (thin lines).  The shading shows differences in scores between the two hemispheres at the forecast ranges indicated.   Currently 3-day, 5-day, 7-day, 10-day forecasts have attained approximately 98.5%, 92%, 80%, 50% anomaly correlation (ACC) respectively.

Current Anomaly Correlation Coefficient (ACC) scores for northern hemisphere, Southern Hemisphere, tropics and Europe are available.

The score for the northern hemisphere extra-tropics is a primary headline score of the ECMWF high resolution forecast (HRES). 

Image Added

Fig6.2.2-4: The plot shows the forecast lead-time (in days) at which Anomaly Correlation Coefficient (ACC) of the HRES forecast dropped below 80% for:

• the month mean (blue line, with blue spots at each month),
• twelve-month mean centred on that month (red line).

Verification follows updated WMO/CBS guidelines as specified in the Manual on the GDPFS, Volume 1, Part II, Attachment II.7, Table F, (2010 Edition - Updated in 2012).