Printable version

The following scores are to be calculated for all parameters against both analysis (except mean sea-level pressure) and observation:

Wind

Mandatory:

  • rms vector wind error
  • mean error of wind speed

Other parameters

Mandatory

  • Mean error
  • Root mean square (rms) error
  • Correlation coefficient between forecast and analysis anomalies (not required for obs)
  • S1 score (only for MSLP and only against analysis)

Additional recommended

  • mean absolute error
  • rms forecast and analysis anomalies (not required for observations)
  • standard deviation of forecast and analysis fields (not required for observations)

Definition

The following definitions should be used
Mean error 

where the sum of the weights

S_w = \sum_{i=1}^n w_i


Root mean square (rms) error

rmse = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (x_f - x_v)_i^2 }


Correlation coefficient between forecast and analysis anomalies

r = \frac{\sum_{i=1}^n w_i (x_f-x_c-M_{f,c})_i (x_v-x_c-M_{v,c})_i}{\sqrt{\left(\sum_{i=1}^n w_i (x_f-x_c-M_{f,c})_i^2 \right) \left(\sum_{i=1}^n w_i (x_v-x_c-M_{v,c})_i^2 \right) }}

rms vector wind error

rmse = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (\vec{V}_f - \vec{V}_v)_i^2 }

Mean absolute error

MAE = \frac{1}{S_w} \sum_{i=1}^n w_i | x_f - x_v |_i

rms anomaly

rmsa = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (x - x_c)_i^2 }

standard deviation of field 

sd = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (x - M_x)_i^2 }

where

M_x = \frac{1}{S_w} \sum_{i=1}^n w_i x_i

S1 score

S1 = 100 \frac{\sum_{i=1}^n w_i (e_g)_i}{\sum_{i=1}^n w_i (G_L)_i}


Where:


x_f = the forecast value of the parameter in question; x_v = the corresponding verifying value; x_c = the climatological value of the parameter; n = the number of grid points or observations in the verification area; M_{f,c} = the mean value over the verification area of the forecast anomalies from climate; M_{v,c} = the mean value over the verification area of the analysed anomalies from climate; \vec{V}_f = the forecast wind vector; \vec{V}_v = the corresponding verifying value;

The differentiation is approximated by differences computed on the verification grid:

e_g = \left ( \left | \frac{\partial}{\partial x}(x_f-x_v)\right | + \left | \frac{\partial}{\partial y}(x_f-x_v)\right | \right ) G_L = \max \left ( \left | \frac{\partial x_f}{\partial x}\right | , \left | \frac{\partial x_v}{\partial x}\right | \right) + \max \left ( \left | \frac{\partial x_f}{\partial y}\right | , \left | \frac{\partial x_v}{\partial y}\right | \right)

The weights w i applied at each grid point or observation location are defined as

  • Verification against analyses: w_i = \cos \theta_i , cosine of latitude at the the grid point i

  • Verification against observations: w_i = 1/n , all observations have equal weight