Page History

...

Charts available are: 2m temperature, surface temperature, total precipitation, mean sea level pressure, winds. 

Colouring on the charts is implies >90% significance for the anomaly.  This does not mean the white shading on the maps necessarily points to:

• a forecast of average.
• a 'don't know' forecast of climatology.

Click the mouse over any location on the ensemble charts to produce probability information in diagrammatic form.  Diagrams available:

...

Measures of skill of forecasts. 

The extended range products should always be used with historical skill metrics in mind.

Broad indications of ensemble performance and predictability in the extended range should be taken into account when considering the extended range output.  Users should consult verification information:

• monthly forecast reliability diagrams (for 2m temperature, surface temperature, total precipitation, mean sea level pressure)
• monthly forecast relative operating characteristics (ROC) diagrams (for 2m temperature, surface temperature, total precipitation, mean sea level pressure).

Relative Operating Characteristics diagrams

The Relative Operating Characteristics give a measure of the effectiveness of a forecast system to predict an event that actually happens balanced against forecast of an event that fails to occur.   ROC is derived from measuring the area beneath the results plotted on a ROC diagram.  ROC area values indicate:

• ROC value 1.0 implies consistent Hit Rate and no False Alarms.
• ROC value 0.5 implies no skill with as many Hits as False Alarms. 
• ROC value 0.0 implies consistent False Alarm rate and no Hits.

...

• below the diagonal implies over-forecasting.  The forecast probability is greater than the observed frequency.
• above the diagonal implies under-forecasting.  The forecast probability is less than the observed frequency.

An explanation of reliability diagrams and ROC diagrams is given in the annex to this guide.

...

The ROCmap shows the ROC score computed over each grid point with a resolution of 2.5 degrees longitude by 2.5 degrees latitude.  The charts are anomalies derived from previous extended range ensemble forecasts compared with ER-M-climate. 

The map colours show values of the ROC score shaded according to the scale above the map:

• Red: ROC > 0.5 - the monthly forecast has more skill than climatology (pink, reds, dark browns).
• Blue: ROC < 0.5 - the monthly forecast has less skill than climatology (cyan, blues, dark blues).

...

Ranked Probability Skill Score (RPSS) chart  

The RPSS map shows the RPSS score computed over each grid point with a resolution of 2.5 degrees longitude by 2.5 degrees latitude.

The Ranked Probability Skill Score (RPSS) compares the Ranked Probability Score of a probabilistic forecast system with a reference of some sort, usually climatology.  

The Ranked Probability Skill Score  charts give a measure of the extended range ensemble forecasts and is a measure of how good forecasts are in matching observed outcomes.  Forecasts falling into the upper or lower terciles of ER-M-climate, are verified against the subsequent reanalysis or operational analysis (for precipitation, 24hr forecasts).  This is equivalent to the Brier skill score where upper and lower terciles are used.

The map colours show values of the RPSS score (equivalent to the Brier skill score) shaded according to the scale above the map:

• RPSS = 1 the forecast has perfect skill compared to the reference (assumed to be climatology) - the forecast is beneficial;
• RPSS = 0 the forecast has no skill compared to the reference - the forecast has no benefit over climatology;
• RPSS = a negative value the forecast is less accurate than the reference  - the forecast is misleading.

...