Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Confirmed.

Printable version

The following scores are to be calculated for all parameters against both analysis (except mean sea-level pressure) and observation:

Wind

Mandatory:

  • rms vector wind error
  • mean error of wind speed

Other parameters

Mandatory

  • Mean error
  • Root mean square (rms) error
  • Correlation coefficient between forecast and analysis anomalies (not required for obs)
  • S1 score (only for MSLP and only against analysis)

Additional recommended

  • mean absolute error
  • rms forecast and analysis anomalies (not required for observations)
  • standard deviation of forecast and analysis fields (not required for observations)

Definition

The following definitions should be used
Mean error 

Mathdisplay
M = \frac{1}{S_w} \sum_{i=1}^n w_i (x_f - x_v)_i

where the sum of the weights

Mathdisplay
S_w = \sum_{i=1}^n w_i


Root mean square (rms) error

Mathdisplay
rmse = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (x_f - x_v)_i^2 }


Correlation coefficient between forecast and analysis anomalies

Mathdisplay
r = \frac{\sum_{i=1}^n w_i (x_f-x_c-M_{f,c})_i (x_v-x_c-M_{v,c})_i}{\sqrt{\left(\sum_{i=1}^n w_i (x_f-x_c-M_{f,c})_i^2 \right) \left(\sum_{i=1}^n w_i (x_v-x_c-M_{v,c})_i^2 \right) }}

rms vector wind error

Mathdisplay
rmse = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (\vec{V}_f - \vec{V}_v)_i^2 }

Mean absolute error

Mathdisplay
MAE = \frac{1}{S_w} \sum_{i=1}^n w_i | x_f - x_v |_i

rms anomaly

Mathdisplay
rmsa = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (x - x_c)_i^2 }

standard deviation of field 

Mathdisplay
sd = \sqrt {\frac{1}{S_w} \sum_{i=1}^n w_i (x - M_x)_i^2 }

where

Mathdisplay
M_x = \frac{1}{S_w} \sum_{i=1}^n w_i x_i

S1 score

Mathdisplay
S1 = 100 \frac{\sum_{i=1}^n w_i (e_g)_i}{\sum_{i=1}^n w_i (G_L)_i}


Where:


Mathinline
x_f

= the forecast value of the parameter in question;

Mathinline
x_v

= the corresponding verifying value;

Mathinline
x_c

= the climatological value of the parameter; n = the number of grid points or observations in the verification area;

Mathinline
M_{f,c}

= the mean value over the verification area of the forecast anomalies from climate;

Mathinline
M_{v,c}

= the mean value over the verification area of the analysed anomalies from climate;

Mathinline
\vec{V}_f

= the forecast wind vector;

Mathinline
\vec{V}_v

= the corresponding verifying value;

The differentiation is approximated by differences computed on the verification grid:

Mathdisplay
e_g = \left ( \left | \frac{\partial}{\partial x}(x_f-x_v)\right | + \left | \frac{\partial}{\partial y}(x_f-x_v)\right | \right )


Mathdisplay
G_L = \max \left ( \left | \frac{\partial x_f}{\partial x}\right | , \left | \frac{\partial x_v}{\partial x}\right | \right) + \max \left ( \left | \frac{\partial x_f}{\partial y}\right | , \left | \frac{\partial x_v}{\partial y}\right | \right)

The weights w i applied at each grid point or observation location are defined as

  • Verification against analyses:

    Mathinline
    w_i = \cos \theta_i

    , cosine of latitude at the the grid point i

  • Verification against observations:

    Mathinline
    w_i = 1/n

    , all observations have equal weight