Background

Advances in artificial intelligence have led to the recent emergence of skillful AI-based and hybrid AI/physically based models for weather prediction applications.  Development and evaluation of the performance of these systems has been undertaken by individual institutions, private corporations and academic researchers.  These efforts have led to the creation of data aggregation and dissemination platforms; however, a model intercomparison project (MIP) with broad international engagement is needed to facilitate development and evaluation activities at national meteorological centres.  As a WMO entity, the Working Group on Numerical Experimentation (WGNE) is well positioned to lead such a global effort.

The WMO Integrated Processing and Prediction System (WIPPS) is a global network of operational centres providing earth-system analyses and predictions to all WMO Members and the wider community. Designated WIPPS centres agree to provide defined sets of mandatory and recommended products to all Members. Developments in AI may have significant implications for operational practices and the evolution of the WIPPS. Artificial intelligence-based systems are significantly cheaper to run than traditional NWP models and bring substantial opportunities as well as potential risks. A priority is to provide guidance to users on the use of AI-based forecasts. An intercomparison of data-driven models and comparison of strengths and weaknesses compared to traditional NWP models will provide essential guidance to WMO Members.

The need for an AI-inclusive MIP was identified by WIPPS as a pilot project proposed to WGNE at its 39th Annual Meeting (Fall 2024).  Discussions at the WGNE meeting revealed broad support for this initiative within the numerical modelling community and laid out the primary objectives for the project.

Development and fine-tuning of AI-based models requires access to significant technical and computing resources, stressing the capacities of smaller operational centres.  Despite these challenges, all members of WGNE agree that broad engagement in an AI-focused MIP is essential, particularly given questions about the performance of AI-based models in regions with sparse observations.  A key project outcome will therefore be guidance for assessments of prediction quality, an objective that will be achievable through the active involvement of the Joint Working Group for Forecast Verification Research (JWGFVR). 

For ease of reference, we hereafter follow the terminology of Radford et al. (2024) in which AI-based models are referred to as AIWP (artificial intelligence weather prediction) systems to distinguish them from physically based NWP models.  Both are specific instances of weather prediction (WP) systems, an umbrella designation that gives rise to the proposed intercomparison project’s name: WP-MIP.

Contributor checklist

Core Contributions

• Run status updated in the Resources table of the project whitepaper
• Complete set of initializations (3-day intervals from 0000 UTC 1 January 2024 to 0000 UTC 29 December 2024, inclusive)
• Forecast fields are provided at 6-hour intervals out to day-10 at as many of the 17 mandatory pressure levels as possible:  1000 hPa, 925 hPa, 850 hPa, 700 hPa, 500 hPa, 400 hPa, 300 hPa, 250 hPa, 200 hPa, 150 hPa, 100 hPa, 70 hPa, 50 hPa , 30 hPa, 20 hPa, 10 hPa, 1 hPa
• Forecasts remapped to global lat/lon 0.25o grid:  1440x721 grid points, including poles
• All forecast hours concatenated into the same file, with separate files for pressure-level and surface-level fields
• Radiative and turbulent fluxes follow a positive-downwards convention
• Precipitation is accumulated since the beginning of the integration
• Units are MKS as specified in the protocol and grib2 conventions
• Forecasts converted to grib2 format
• Metadata provided for MARS archiving
• OIC Stream only:  Final analyses provided in same format as forecasts (0.25o grid, pressure level, grib2) at 6-hour intervals for the full study period (0000 UTC 1 January 2024 to 0000 UTC 8 January 2025) 

SP1:  Tropical Cyclones

• Inform subproject lead (Masashi Ujiie) of intent to submit contribution
• Complete set of initializations (filling in 0000 UTC initializations not run for core contributions)
• Sea level pressure and 10-m winds at 6-hourly intervals for 10-day forecasts

SP2:  Sub-seasonal Predictions

• Inform subproject lead (Debbie Hudson) of intent to submit contribution
• Integrations for core contributions extended to 15-day lead times

OIC Stream only: Extend set of final analyses provided in the same format as forecasts (0.25o grid, pressure level, grib2) at 6-hour intervals to cover final 15-day forecasts (5 additional days:  0000 UTC 8 January 2025 to 0000 UTC 13 January 2025)