Page History

...

The difficulty in producing an accurate forecast rises with forecast lead-time.  Both broad scale accuracy and fine scale variability are desirable.  However, both cannot be captured at the same time with a single forecast at longer time ranges.  In the case of less predictable weather situations, smoother forecasts (such as the ensemble mean) provide on average lower forecast errors.  ML-based forecasts show less energy at smaller scales and patterns ack lack detail.  This loss of energy at synoptic scales is particularly strong at day 6 and day 10.

...

Fine-tuning is a strategy used to mitigate a change in the IFS model (from Cy49 to Cy50) used in the analyses and re-analyses.  These in turn affect the learning process for the AIFS models.  Fine-tuning for AIFS involves adapting pre-trained weights to specific regions, higher resolutions, or new physical variables. Also, further training on forecast analyses (rollout) improves location accuracy in long-term forecasts but suppresses fine-scale variability and smoother forecasts.  Thus there is a trade off between accuracy of location, or in the detail of depth or complexity of a forecast system.   This is particularly important in forecasting tropical cyclones.

Rollout

Rollout training is a strategy to improve accuracy of long-term forecasts, particularly with tropical cyclones.  During training the model predicts multiple time-steps in the future and minimises errors against ERA5.  The period into the future gradually is increased from 12 to 72 hours ahead.  The accumulation of errors is reduced but this suppresses fine-scale variability and so the lower predictability produces smoother forecasts.  Distinguishing pre-training effects from smoothing induced by rollout is therefore essential.

A rollout procedure is common among state-of-the-art MLWP models.  

Image Added

Fig2B.5-1: Forecast accuracy–activity trade-off for geopotential height at 500 hPa in the northern hemisphere.  The plot shows the forecast relative accuracy with respect to the IFS control forecast versus the forecast relative activity with respect to the IFS analysis.  Results for forecasts at lead time day 1 (dot) up to date 10 (squares) are plotted.  Accuracy is here measured with the anomaly correlation coefficient (ACC).  Fine-tuning with rollout (orange) shows reduction in activity (i.e. fine detail of smaller scales and patterns) but a large increase in accuracy.  Fine-tuning without rollout (green) shows an increase in activity but only a small increase in accuracy.  Without fine tuning (blue) there is a reduction in both activity and accuracy. 

Image Added

Fig2B.5-2: An example of the effect of using fine-tuning with and without rollout.  Mean sea-level pressure fields associated with Tropical Cyclone Mocha initialised 12UTC 9 May 2023 , valid 12UTC 13 May 2023.  With rollout (A) the forecast cyclone circulation lacks detail and in particular a low central pressure.  Without rollout (B) the forecast cyclone circulation shows realistically low central pressure. The IFS forecast cyclone circulation is shown for comparison.  The relative ability in accuracy of forecast location is not shown.

Parameters forecast by AIFS Single v2 and AIFS ENS v2

...

Additional Sources of Information

(Note: In older material there may be references to issues that have subsequently been addressed)

More detail available on training strategies for tropical cyclone forecasts.

...