...
The AI Weather Quest is a collaborative forecasting challenge hosted by ECMWF, designed to explore the potential of AI and machine learning in medium-range weather prediction. Participating teams are challenged with providing quintile-based probabilistic forecasts for forecasts for three key variables: near-surface air (2m) temperature (tas), mean sea level pressure (mslp), and accumulated precipitation (pr). Forecasts target lead times of days of days 19 to 25 (week 3) and days and days 26 to 32 (week 4).
The Leaderboards page displays ranked Probability Skill Scores (RPSS) for submitted forecasts, as well as team and model rankings, through leaderboards and evolution graphs.
...
There are two main types of RPSS leaderboard (buttons before the leaderboards allow you to switch between the following two views):
1. Period-aggregated RPSS leaderboard
- Shows RPSS scores aggregated over a selected competitive period (e.g. SON 2025)
- Useful for assessing overall performance across multiple weeks
- A team’s score will only appear in the Period-aggregated RPSS leaderboard if forecasts have been submitted up until the chosen competitive week.
2. Weekly RPSS leaderboard
- Shows RPSS scores for a specific competition week
- Useful for tracking short-term performance
For further details on how scores are calculated, see the evaluation system page of the AI Weather Quest website.
Exploring team profiles
Click on any team name in the leaderboard to view:
- Team members (if public)
- Model descriptions
- Participation history (forecast submissions by week, window, and variable)
Evolution graphs
Each RPSS table is accompanied by two evolution graphs:
...
- X-axis: competition week numbers
- Y-axis: RPSS scores of models
- Default view: top 5 models for selected filters
- Up to 10 models can be selected at once
Exploring team profiles
Click on any team name in the leaderboard to view:
...
Note: Line colours in the team and model graphs are assigned independently, so a team’s line colour will not necessarily match the colour of its models.