Selection of stations

GloFAS forecast skill assessment is conducted on all GloFAS v5 calibration points.

Background on GloFAS forecasting

The forecast quality of GloFAS river discharge forecasts (out to 15 day lead time) has been is evaluated in order to provide additional information to users to aid support decision-making. The forecast evaluation method is described in Harrigan et al. (2020a2023) and summarised below with the . The results provided in the following page:  GloFAS v4 forecast skill as well as summarised as a new headline score included as a .A headline forecast skill score is also provided in the "Forecast skill" layer on of the GloFAS web map viewer : (see GloFAS forecast skill product).

Headline forecast skill score 

The headline forecast skill score is the maximum lead time (in days), up to 15 - days ahead, in for which the Continuous Ranked Probability Skill Score (CRPSS) is greater than a value of 0.5, when compared to a persistence or climatology benchmark forecast using GloFAS -ERA5 historical v5 river discharge reanalysis (also known as as Forced simulation (sfo) within within CEMS)  as as proxy observations (Harrigan et al., 2020b). . Forecast skill is calculated using 20 years of river discharge reforecasts for a set of past dates, based on , generated with a configuration as close as possible to the operational settingforecasting system. ECMWF-ENS medium and extended -range reforecasts are used and are run twice per week for the past 20-years with 11 ensemble membersinitialised on fixed calendar dates (1, 5, 9, 13, 17, 21, 25 and 29 of each month, excluding 29 February), with 11 ensemble members.

For GloFAS v5, forecast skill is calculated separately for each month and updated on the first day of every month, using the latest available reforecasts. Each monthly calculation uses reforecasts initialised over a four-week period spanning the previous and current month, over the 20-year reforecast period. This approach captures seasonal variations in forecast skill while ensuring that the evaluation reflects the latest operational forecasting system.

Scores are shown on the GloFAS map viewer in the 'Forecast skill' layer under the 'Evaluation' menu (an example of the layer is shown here: GloFAS forecast skill product). For each each GloFAS web reporting v5 calibration point, the maximum lead time the CRPSS is greater than 0.5 is given, with stations with purple circles having high skill for indicating skill above the threshold at longer lead times. The category "0", marked as light pink circles represents stations that have forecast skill lower than the points where CRPSS does not exceed 0.5 threshold for at any lead time. Note: This does not mean that a station has no skill. Only when the CRPSS ≤ 0 is when the forecast has no skill when compared to a persistence or climatology benchmark forecast. 

Method

Reforecasts

ECMWF-ENS medium and extended-range reforecasts are generated on fixed calendar dates (1, 5, 9, 13, 17, 21, 25 and 29 of each month, excluding 29 February), for the corresponding dates in the previous 20 years, with 11 ensemble members. These meteorological reforecasts are used to force the GloFAS hydrological modelling chain, producing 20 years of river discharge reforecasts at a daily time step, with a forecast horizon of 46 days. Only the first 15 days are used for the medium-range forecast skill evaluation.

For GloFAS v5, forecast skill is calculated separately for each month, using reforecasts initialised over a four-week period spanning the previous and current month, across the 20-year reforecast period. As ECMWF reforecasts are made available up to 20 days ahead of their nominal initialisation dates, reforecasts for the first part of the current month are already available when the skill is calculated on the first day of the month. These are combined with reforecasts from the previous month to provide approximately one month of data, as representative as possible of the month being evaluated.

The skill layer is recalculated on the first day of every month using the latest available reforecasts, ensuring that the evaluation reflects the current operational forecasting system and its seasonal variations in forecast performance. Figure 1 illustrates the reforecast configuration for the skill layer calculated on 1 November 2026.

Image Added are generated every Monday and Thursday, for the same date in the past 20 years for 11 ensemble members out to a lead time of 30 days. For the GloFAS forecast skill evaluation, ECMWF-ENS reforecasts run over a year long reference period (for example, January to December 2019 for GloFAS version 2.2) are used. These are then forced through the GloFAS hydrological modelling chain to produce 20 years of river discharge reforecasts, twice weekly, for 11 ensemble members. A schematic of the ECMWF-ENS reforecast configuration is given in Figure 1 for the reference period January to December 2019. In total, there are 2080 start dates in a GloFAS reforecast set (52 weeks x 2 per week x 20 years). GloFAS river discharge reforecasts are run for each river cell with an upstream area > 1000 km² at a 24 hr time-step out to a lead time of 15 days with 11-ensemble members each.Image Removed

Figure 1: ECMWF-ENS reforecast configuration schematic for the reference period January to December 2019for the GloFAS v5 monthly forecast skill evaluation, illustrating the reforecasts used for the skill layer calculated on 1 November 2026, using 20 years of reforecasts (2006–2025). 

Benchmark forecasts

Following Pappenberger et al. (2015), and because GloFAS produces seamless forecasts across short and medium lead times (

days 1 to 15), two benchmarks are considered

, each calculated for

GloFAS

v5 calibration points: persistence, typically used for short lead times where the forecast signal is dominated by serial correlation of river discharge, and climatology, typically used for longer lead times where the forecast signal is dominated by the seasonality of river discharge. These are defined as follows:

• persistence Persistencebenchmark forecast defined :Defined as the single GloFAS -ERA5 v5 daily river discharge of reanalysis value for the day preceding the reforecast start date. The same river discharge value is used for all lead times.
• climatology Climatology benchmark forecast based :Based on a 20-year climatological sample (2003-2022) of moving 31-day windows of GloFAS -ERA5 v5 river discharge reanalysis values, centred on the date being evaluated (+- 15 ±15 days). From each 1240620-valued climatological sample (i.e. 20 years × 31 -day windowdays), 11 fixed quantiles (Qn) at 10 % 10% intervals were are extracted (Q0, Q10, Q20, Q30, Q40, Q50, Q60, Q70, Q80, Q90, Q100). The fixed quantile climate distribution used therefore resulting climatological distribution varies by lead time, capturing the temporal variability in local river discharge climatology.

Proxy observations

Forecast skill is evaluated

against GloFAS v5 river discharge reanalysis, used as a proxy

for river discharge observations,

at GloFAS v5 calibration points. Using reanalysis instead of in situ

observations

allows forecast skill

to be assessed while reducing the influence of hydrological model errors. It also provides consistent spatial and temporal coverage

across the

GloFAS domain.

Users

should be aware that

a key assumption

of the proxy observation approach is that the GloFAS

v5 reanalysis reasonably represents the hydrological behaviour at the location of interest. If

hydrological model performance is poor,

forecast skill scores

should be interpreted with caution. Further information on GloFAS v5 hydrological model performance is available in the GloFAS v5 calibration hydrological model performance documentation.

Skill score

The ensemble forecast performance is evaluated using the Continuous Ranked Probability Score (CRPS) (Hersbach, 2000), one of the most widely used headline scores for probabilistic forecasts. The CRPS compares the continuous cumulative distribution of an ensemble forecast with the distribution of the observations. It has an optimum value of 0 and measures the error in the same units as the variable of interest (here river discharge in m³ s^-1). It collapses to the mean absolute error for deterministic forecasts (as is the case here for the single-valued persistence benchmark forecast). The CRPS is expressed as a skill score to calculate forecast skill, CRPSS, which measures the improvement over a benchmark forecast and is given in:

{CRPSS}={1-}\frac{{CRPS_{fc}}}{{CRPS_{bench}}}

A CRPSS value of 1 indicates a perfect forecast, CRPSS > 0

indicates forecasts more skilful than the benchmark, CRPSS = 0

indicates forecasts are

as

skilful as the benchmark, and

CRPSS < 0

indicates forecasts

less skilful than

the benchmark

. The headline GloFAS forecast skill score uses a CRPSS threshold of 0.5 in the summary layer

of the GloFAS web map viewer

. A CRPSS of 0.5 indicates a 50% reduction in CRPS relative to the benchmark forecast.

The CRPSS is calculated

using GloFAS v5 river discharge reforecasts against both persistence and climatology benchmark

forecasts, with GloFAS v5 river discharge reanalysis as proxy observations

. Scores are calculated separately for each month and updated monthly using the latest available reforecasts. The headline scores are

displayed at GloFAS v5 calibration points on the GloFAS map viewer,

with detailed CRPSS and CRPS

plots

available for individual points.

References

References

Harrigan, S.,

Zoster

Zsoter, E., Cloke, H., Salamon, P., and Prudhomme, C.,

2020a

2023. Daily ensemble river discharge reforecasts and real-time forecasts from the operational Global Flood Awareness System

, Hydrol

.

Earth Syst. Sci. Discuss., https://doi.org/10.5194/hess-2020-532, in review.Harrigan, S., Zsoter, E., Alfieri, L., Prudhomme, C., Salamon, P., Wetterhall, F., Barnard, C., Cloke, H., and Pappenberger, F., 2020b. GloFAS-ERA5 operational global river discharge reanalysis 1979–present, Earth Syst. Sci. Data, 12, 2043–2060,

Hydrology and Earth System Sciences, 27, 1–19. https://doi.org/10.5194/

essd

hess-

12

27-

2043

1-

2020.

2023

Hersbach, H., 2000. Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems.

Wea.

Weather and Forecasting, 15, 559–570. https://journals.ametsoc.org/view/journals/wefo/15/5/1520-0434_2000_015_0559_dotcrp_2_0_co_2.xml 

Pappenberger, F., Ramos, M.H., Cloke, H.L., Wetterhall, F., Alfieri, L., Bogner, K., Mueller, A., and Salamon, P., 2015. How do I know if my forecasts are better? Using benchmarks in hydrological ensemble prediction. Journal of Hydrology, 522, 697–713. https://doi.org/10.1016/j.jhydrol.2015.01.024