The AI forecasting process

Essentially at each grid point, the forecasting process applies the algorithms produced by the ML training to forecast each variable.  It uses the complete set of available observed variables and produces a complete set of forecast variables for six hours later.  Two AI forecast systems are used:

• AIFS Single which is the ECMWF stand alone AI forecast.
• AIFS-ENS which is the ECMWF ensemble of AI forecasts.

Making a forecast with AI is very efficient.  It requires only a single Graphics Processing Unit (GPU), takes less than a minute to run, and consumes a tiny fraction of the energy required for an IFS forecast.  This brings the prospect of more frequent and/or quite large ensembles of AI forecasts.

Multi-date verification suggests AI broadscale forecasts score better than classical NWP.   However, shorter wave length features and fine detail is not well captured, particularly as forecast lead time increases.

Fig2.1.6.2-1: Forecasting process using AI for a single parameter.  The algorithm to produce each single parameter uses all the set of input variables.  The algorithms relating the observed data to predicted value of each parameter six hours later have been derived by ML.

Fig2.1.6.2-2: Forecasting process using AI for all the parameters.   Each algorithm to produce each output variable uses all input variables.  The algorithms relating observed data to predicted data six hours later have been derived by ML.

Table1: Observed and forecast variables and constants at pressure levels used by the machine learning process and forecast process within ECMWF AIFS.  AIFS receives as input a representation of the state of the atmosphere - from ERA5 for the ML process; from the ECMWF operational analysis for the AI forecast process.  Both ML during the training process and AI during the forecast process predict the atmospheric state for six hours in the future.  Currently AIFS only uses data at the surface and at standard pressure levels (diagram on the right).

Strengths of using AI are:

• it gives very rapid results.  The observation to output relationships are quite simple and require little processing power at each grid point.  As each time step is six hours, few iterations are needed for a forecast.   This brings the prospect of very large or very frequent ensembles of AI forecasts.
• it is very cheap to run, because of the simplicity of the relationship program.  
• output can be tailored directly to uses other than meteorology (e.g. to health services without any knowledge of medical theory after a ML training process has been performed using health service parameters).

Weaknesses of using AI are:

• there is no need for comprehensive understanding of physics theory.  The relationships are simple to program but at the expense of understanding physically the meteorological effects that are in action.  
• the AI process is effectively a "black box" producing results by a process unknown to the user.   It requires a good deal of trust in the method though initial results show high effectiveness.  The user can have difficulty interpreting or explaining forecast results.
• the ability to interpret the results of AI forecasts ("Interpretability") may be built up with experience; the ability to explain the results of AI forecasts ("Explainability") may be more difficult.
• the set of observed and forecast variables is limited (see Table1).
• post-processing at a given location may require further physical or practical interpretation.
• problems regarding inter-compatabilty between programing languages of physical and AI models.  This might become a problem where hybrid models are employed (e.g. where transferring data from AI to physical model for post-processing).
• input observations at different times and locations have to be assigned to specific grid points (encoding) and the reverse process to assign forecast values from grid points to specific locations (decoding).
• each forecast variable is independent of the others.  They are not interdependent.  The forecast wind may not be consistent with the forecast pressure or height gradient.  

(FUG Associated with Cy49r1)