Selection of stations

GloFAS forecast skill assessment is conducted on all GloFAS v5 calibration points.

Background on GloFAS forecasting

The forecast quality of GloFAS river discharge forecasts (out to 15 day lead time) is evaluated in order to provide additional information to users to support decision-making. The forecast evaluation method is described in Harrigan et al. (2023) and summarised below. The results provided in the following page: GloFAS v4 forecast skill. A headline forecast skill score is also provided in the "Forecast skill" layer of the GloFAS web map viewer (see GloFAS forecast skill product).

Headline forecast skill score 

The headline forecast skill score is the maximum lead time (in days), up to 15 days ahead, for which the Continuous Ranked Probability Skill Score (CRPSS) is greater than a value of 0.5, when compared to a persistence or climatology benchmark forecast using GloFAS v5 river discharge reanalysis (also known as Forced simulation (sfo) within CEMS) as proxy observations. Forecast skill is calculated using 20 years of river discharge reforecasts, generated with a configuration as close as possible to the operational forecasting system. ECMWF-ENS medium-range reforecasts are initialised on fixed calendar dates (1, 5, 9, 13, 17, 21, 25 and 29 of each month, excluding 29 February), with 11 ensemble members.

For GloFAS v5, forecast skill is calculated separately for each month and updated on the first day of every month, using the latest available reforecasts. Each monthly calculation uses reforecasts initialised over a four-week period spanning the previous and current month, over the 20-year reforecast period. This approach captures seasonal variations in forecast skill while ensuring that the evaluation reflects the latest operational forecasting system.

Scores are shown on the GloFAS map viewer in the 'Forecast skill' layer under the 'Evaluation' menu (an example of the layer is shown here: GloFAS forecast skill product). For each GloFAS v5 calibration point, the maximum lead time the CRPSS is greater than 0.5 is given, with purple circles indicating skill above the threshold at longer lead times. The category "0", marked as light pink circles represents points where CRPSS does not exceed 0.5 at any lead time. Note: This does not mean that a station has no skill. Only when the CRPSS ≤ 0 is when the forecast has no skill when compared to a persistence or climatology benchmark forecast. 

Method

Reforecasts

ECMWF-ENS medium and extended-range reforecasts are generated on fixed calendar dates (1, 5, 9, 13, 17, 21, 25 and 29 of each month, excluding 29 February), for the corresponding dates in the previous 20 years, with 11 ensemble members. These meteorological reforecasts are used to force the GloFAS hydrological modelling chain, producing 20 years of river discharge reforecasts at a daily time step, with a forecast horizon of 46 days. Only the first 15 days are used for the medium-range forecast skill evaluation.

For GloFAS v5, forecast skill is calculated separately for each month, using reforecasts initialised over a four-week period spanning the previous and current month, across the 20-year reforecast period. As ECMWF reforecasts are made available up to 20 days ahead of their nominal initialisation dates, reforecasts for the first part of the current month are already available when the skill is calculated on the first day of the month. These are combined with reforecasts from the previous month to provide approximately one month of data, as representative as possible of the month being evaluated.

The skill layer is recalculated on the first day of every month using the latest available reforecasts, ensuring that the evaluation reflects the current operational forecasting system and its seasonal variations in forecast performance. Figure 1 illustrates the reforecast configuration for the skill layer calculated on 1 November 2026.

Figure 1: ECMWF-ENS reforecast configuration schematic for the GloFAS v5 monthly forecast skill evaluation, illustrating the reforecasts used for the skill layer calculated on 1 November 2026, using 20 years of reforecasts (2006–2025). Graphic design enhanced with AI assistance

Benchmark forecasts

Following Pappenberger et al. (2015), and because GloFAS produces seamless forecasts across short and medium lead times (days 1 to 15), two benchmarks are considered, each calculated for GloFAS v5 calibration points: persistence, typically used for short lead times where the forecast signal is dominated by serial correlation of river discharge, and climatology, typically used for longer lead times where the forecast signal is dominated by the seasonality of river discharge. These are defined as follows:

• Persistence benchmark forecast: Defined as the GloFAS v5 daily river discharge reanalysis value for the day preceding the reforecast start date. The same river discharge value is used for all lead times.
• Climatology benchmark forecast: Based on a 41-year climatological sample (1975-2015 inclusive) of moving 31-day windows of GloFAS v5 river discharge reanalysis values, centred on the date being evaluated (±15 days). From each 1,271-valued climatological sample (41 years × 31 days), 11 fixed quantiles (Qn) at 10% intervals are extracted (Q0, Q10, Q20, Q30, Q40, Q50, Q60, Q70, Q80, Q90, Q100). The resulting climatological distribution varies by lead time, capturing the temporal variability in local river discharge climatology.

Proxy observations

Forecast skill is evaluated against GloFAS v5 river discharge reanalysis, used as a proxy for river discharge observations, at GloFAS v5 calibration points. Using reanalysis instead of in situ observations allows forecast skill to be assessed while reducing the influence of hydrological model errors. It also provides consistent spatial and temporal coverage across the GloFAS domain.

Users should be aware that a key assumption of the proxy observation approach is that the GloFAS v5 reanalysis reasonably represents the hydrological behaviour at the location of interest. If hydrological model performance is poor, forecast skill scores should be interpreted with caution. Further information on GloFAS v5 hydrological model performance is available in the GloFAS v5 calibration hydrological model performance documentation.

Skill score

The ensemble forecast performance is evaluated using the Continuous Ranked Probability Score (CRPS) (Hersbach, 2000), one of the most widely used headline scores for probabilistic forecasts. The CRPS compares the continuous cumulative distribution of an ensemble forecast with the distribution of the observations. It has an optimum value of 0 and measures the error in the same units as the variable of interest (here river discharge in m³ s^-1). It collapses to the mean absolute error for deterministic forecasts (as is the case here for the single-valued persistence benchmark forecast). The CRPS is expressed as a skill score to calculate forecast skill, CRPSS, which measures the improvement over a benchmark forecast and is given in:

A CRPSS value of 1 indicates a perfect forecast, CRPSS > 0 indicates forecasts more skilful than the benchmark, CRPSS = 0 indicates forecasts are as skilful as the benchmark, and CRPSS < 0 indicates forecasts less skilful than the benchmark. The headline GloFAS forecast skill score uses a CRPSS threshold of 0.5 in the summary layer of the GloFAS web map viewer. A CRPSS of 0.5 indicates a 50% reduction in CRPS relative to the benchmark forecast.

The CRPSS is calculated using GloFAS v5 river discharge reforecasts against both persistence and climatology benchmark forecasts, with GloFAS v5 river discharge reanalysis as proxy observations. Scores are calculated separately for each month and updated monthly using the latest available reforecasts. The headline scores are displayed at GloFAS v5 calibration points on the GloFAS map viewer, with detailed CRPSS and CRPS plots available for individual points.

References

Harrigan, S., Zsoter, E., Cloke, H., Salamon, P., and Prudhomme, C., 2023. Daily ensemble river discharge reforecasts and real-time forecasts from the operational Global Flood Awareness System. Hydrology and Earth System Sciences, 27, 1–19. https://doi.org/10.5194/hess-27-1-2023

Hersbach, H., 2000. Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems. Weather and Forecasting, 15, 559–570. https://journals.ametsoc.org/view/journals/wefo/15/5/1520-0434_2000_015_0559_dotcrp_2_0_co_2.xml 

Pappenberger, F., Ramos, M.H., Cloke, H.L., Wetterhall, F., Alfieri, L., Bogner, K., Mueller, A., and Salamon, P., 2015. How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction. Journal of Hydrology, 522, 697–713. https://doi.org/10.1016/j.jhydrol.2015.01.024