Machine Learning Process for AIFS ENS

The algorithms for AIFS ENS are developed rather differently from those for use by AIFS single. 

For the ensemble, it is necessary also to include some form of uncertainty during evaluation of the algorithms.  To do this, white noise is injected into the neural network during the training phase.  The model learns to shape this noise to capture the uncertainty in future weather conditions, so that in the forecast phase when new white noise is injected, the model can create a well-calibrated ensemble.

A small ensemble group (ECMWF uses a group of 4) is used that give information on the variability of these independent results and to introduce a measure of model uncertainty.   At each grid point the set of observed data is processed using four different sets of random weighting functions for each parameter.  The four forecast results are then compared with verifying data.  The error metric (loss function) is the "almost fair CRPS" which measures how good forecasts are.  CRPS is evaluated for the results of the four forecasts.  The CRPS influences what is fed back to the ML processors (back propagation).  This induces modification of the weighting functions for each parameter and the resulting forecasts are compared with the verifying data giving new CRPS values.  This process is repeated many times with the aim to progressively minimise the error metric (CRPS).  When this minimisation is reached the set of weights for each parameter forms the algorithm for the ensemble forecasting process (See Fig2B.4-1).  Using CRPS as a loss function accounts for the limitations of using a finite number of ensemble members  and ensures an accurate and well-calibrated distribution.   Model uncertainty is incorporated as a learnt aspect due to the insertion of white noise.

Fig2B.4-1: Machine learning training process in deriving an algorithm for use in AIFS ENS.  A range of observed data is processed using four different sets of random weighting functions for each parameter.   White noise is introduced to each iteration to emulate model uncertainty.  The four forecast results are then compared with verifying data.  CRPS, which measures how good forecasts are, is evaluated for the results of the four forecasts.  The CRPS influences what is fed back to the processor (back propagation). This induces modification of the weighting functions for each parameter and the resulting forecasts are compared with the verifying data again giving new CRPS value.  Iteration continues until the CRPS is minimised and the set of weights for each parameter becomes the algorithm used by all ensemble members for the forecasting process.  Note: In the diagram "other parameters" includes 6hr rainfall.

Forecasting process of AIFS ENS

Ensemble forecasting (AIFS ENS) brings an ability to quantify uncertainty and to identify possible extremes in the evolution.  An ensemble of the AI forecast procedure can be executed because AI forecasting is rapid and cheap to run. At present AIFS ENS has 50 members and is run at six hour intervals.   However there is potential for much larger and/or more frequent ensembles.  Essentially each ensemble member is similar to AIFS single.

Observations are assimilated using 4D-var to IFS grid point values over the whole globe with resolution about 9km.  This is the starting point for the ensemble control member.  The initial conditions for the other members of the AI ensemble are the IFS re-centered Ensemble Data Assimilation (EDA) with singular vectors as with the perturbations in the IFS ensemble.   However, the perturbations applied to each ensemble member are derived differently, and have different values to those applied to the physical models.  Strictly, there is no unperturbed ensemble control member as appears in the IFS ensemble.  This is because all ensemble members, including the control member, are generated probabilistically.  So, in that sense, they are statistically equivalent.

Currently the AI procedures use grid points at a lower resolution about 0.25 of a degree (~25km).  The IFS analysed data is "encoded" to values at the AI grid point.  These data are then fed to the AI procedure that uses the algorithms derived by the ML trained by minimising CRPS.   The algorithm is not complicated and consequently has fast execution allowing a complete global forecast to be made much faster than IFS and other physics-based models.  Forecasts are made in six hour time steps and output from each time step is used as the input for the next step.  Currently the AI forecasts use 60 repeated six hour time steps to extend the forecast to T+360.   Intermediate forecast data is only available at the simulated six hour time steps.  Currently no information is available at smaller intervals than six hours.

There is only one set of algorithms produced by the ML process for the ensemble and this is used throughout the forecast and by all ensemble members.  Each ensemble member uses the same set of algorithms as the others.  The spread in the ensemble members is generated only by the difference in the initial conditions (perturbations) and the white noise injected into the model.

The limitation in horizontal and vertical resolutions mean that:

• Interpretation is necessary before using any forecast product.
• it is only possible to derive a few post-processed products (e.g. precipitation and convective precipitation).

Making a forecast with AI is very efficient.  It requires only a single Graphics Processing Unit (GPU), takes less than a minute to run, and consumes a tiny fraction of the energy required for an IFS forecast.  This brings the prospect of more frequent and/or quite large ensembles of AI forecasts.

Fig2B.4-2: Schematic illustrating AIFS ENS.  50 members have perturbations based on Ensemble Data Assimilation (EDA) with singular vectors as with IFS ENS.  The control is not truly unperturbed, as the algorithms have been generated probabilistically .

Points to consider when using AIFS ENS output

In addition to points for consideration outlined in the AIFS single section

AIFS ENS forecasts:

• High-quality ensemble forecasts that outperform ECMWF’s traditional physics-based ensemble system in medium-range forecasts on most accuracy and reliability scores.
• Very fast and economical to produce a forecast sequence.
• The principle for construction of meteograms and charts is very similar to that used by IFS ensembles.
• Plots of TCs probability, wind speed and central pressure can show disconcerting gaps.  This is being investigated. 
• Individual ensemble members do not exhibit the smoothing seen in AIFS SIngle and have similar levels of forecast activity to IFS.  This is because the CRPS loss function does not encourage smoothing.
• AIFS ENS control verifies better than AIFS ENS members (possibly with 6-12h gain).
• Are currently little bit over-dispersive.
• Very small totals can appear in the precipitation fields far too often.  This will be fixed in upcoming cycles.
• Physical consistency in the output is unlikely to be as good as with IFS.  This is being investigated.
• Cloud fields appear very "blocky" after T+0.  This is being investigated.

Practical use of AIFS ENS output

Forecasters should consider:

• When a variable level is nominally underground - e.g. for mean sea level pressure in very mountainous areas, or 1000hPa or even 850hPa temperature in very mountainous areas, extreme unrealistic values can develop in the course of a forecast, in some or all ensemble members.  In turn this can manifest as extreme ensemble spread in that particular variable.  One problem area is the Andes, but there are others too. These 'glitches' do not always happen, and their onset is not always at the same lead time (e.g. Fig2B.4-3).
• Some extreme weather events appear less intense than IFS ensemble (e.g. windstorms).
• Temperature extremes are more consistently handled, generally beating classical NWP forecasts.
• The AIFS ENS control has a slight advantage over other AIFS ENS members because it starts from unperturbed initial conditions.  However, because of the ML training it contains model uncertainty.  It is not the same as IFS ensemble control (Ex-HRES).
• Extremes are better identified than with AIFS-Single.
• AIFS ENS (and IFS ENS) has some difficulty handling easterly waves over central Africa. 

Information on issues associated with AIFS ENS is given in Known AIFS ENS Forecasting Issues  

The dissemination schedule is given in Section 3.1:

• AIFS schedules and products.

Fig2B.4-3: An example of an AIFS ENS meteogram for Riga DT06UTC 02 Oct 2025.  The plots are in standard box and whisker form.   The AIFS ENS control (Red) and AIFS Single (Blue) are shown as continuous lines.  Note AIFS single departs largely outside the AIFS ENS box and whisker in places (e.g. precipitation).  This may be due to the difference in the ML derivation of the algorithms for AIFS single and AIFS ENS.

FIG2B.4-4: An example of AIFS ENS chart output from AIFS ENS mean and spread VT12UTC 30 Jun 2025, DT12UTC 24 Jun 2025.  Large uncertainty over the southern Andes as high as 8-14C locally. This can be due to 1000hPa contour lying below the model surface in mountainous areas.

(FUG Associated with Cy50r1)