Model Data Assimilation, 4D-Var
The four-dimensional variational analysis (4D-Var) system uses an optimisation procedure whereby the initial condition is adjusted to obtain an optimal fit through all the observations in the assimilation time window and at the same time tries to stay as close as possible to the first guess. 4D-Var uses the concept of a continuous feedback between observations and IFS model data analysis. The impact of the observations is determined by three considerations:
- the assumed accuracy of the observations, considered to be more or less static. However, consistently poor observations can be blacklisted.
- the representativeness of the observation relative to the IFS model's depiction (i.e. taking into account the differences between the altitude of the observation and the corresponding altitude of that point within the orography within the IFS, or the differences between the location of the observation and the grid point).
- the accuracy of the short-range forecasts, which are flow-dependent. Hence the uncertainty may be larger in a developing baroclinic depression than in a subtropical high-pressure system.
The 4D-Var analysis uses the IFS dynamics and physics to create a sequence of states that fits as closely as possible to the available observations. These states are consistent with the dynamics and physics of the atmosphere, as expressed by the equations of the IFS model. 4D-Var compares the actual observations with that which would be expected at the time and position of the observation given the model fields - it is in effect a short-range forecast that serves to bring information forward from the previous cycle. The analysis essentially uses data within a time window, currently 09–21UTC for the 12UTC model run; 21–09UTC for the 00UTC model run. However in practice for the main forecasts of HRES and ENS the data cut-off used (i.e. the last time) is brought back to 15UTC and 03UTC respectively for the 12UTC and 00UTC runs, in order to be able to deliver forecasts to customers in a timely manner (see the continuing sequence of analyses). All observational data are processed similarly, including radiances from satellites. Since the introduction of cycle 45r1 in June 2018 a weakly coupled sea-ice atmosphere assimilation has been used in the surface analyses of the HRES 4D-Var, and in the ensemble of data assimilations (EDA); previous to this a remotely-generated sea ice cover analysis (OSTIA) had been used directly.
4D-Var at ECMWF is based on an incremental and iterative approach to minimising a cost function - in effect minimizing the departure of the final adjusted analysis from the observed values and the last available short range forecast. This takes place within a set of nested loops. The inner loop has low spatial resolution (TL255 L137) and produces preliminary low-resolution analysis increments using full linearised physics (except the first inner loop of the EDA). By iterating forwards and backwards in time the analyses can be adjusted towards the observations and induce consistent adjustments with other variables. Subsequent loops are at higher resolution (TL319 L137 and TL399 L137) with the same full linearized physics. This incremental approach provides considerable flexibility in the use of computer resources.
The resulting adjustment of a variable generates physically and dynamically consistent corrections of other variables. Thus a sequence of observations of humidity from a satellite infrared instrument that shows a displacement of atmospheric structures will entail a correction not only of the moisture field but also of the wind and temperature fields. For satellite radiances the variational scheme modifies the fields of temperature, wind, moisture and also ozone in such a way that the simulated observations are brought closer to the observed values.
To estimate the flow-dependent uncertainty, a set of 3-hour forecasts, valid at the start of the 4D-Var time-window, is computed from 25 perturbed, equally likely analyses. Small variations are imposed on the observations and the sea surface temperature to reflect uncertainties, and also within the error parameterisation to cope with uncertainties in the forecast evolution. The perturbations produced using this Ensemble of Data Assimilations (EDA) are also used for the construction of the perturbations in the forecast ensemble.
Fig2.5.1A(left): The ECMWF 4-dimensional data assimilation system determines a correction to the background initial condition (blue line) that leads to an analysis that is somewhere "midway" between background and observations. In simplest terms the analysis is a weighted mean of the background and observations.
Fig2.5.1B(right): Each observation has an error (instrumental, representativeness, etc) and error within the IFS forecast models is also taken into account. A way to simulate both these effects is to run an Ensemble of Data Assimilations (EDA). These are shown in green.
Analogies to 4D-Var and the EDA
Suppose a forecaster wants to create a sequence of manually analysed hourly synoptic charts for their region, which evolve smoothly, continuously and realistically from one time to the next, throughout. To achieve this one approach would be to draw up one chart, using all available data, surface observations, imagery etc, then another for a subsequent time in the same way. Then the forecaster might go back to the first chart, rub something out and re-draw it in a rather different fashion, that still fits the available observations reasonably well, but that allows the next chart to follow on better. Then they might draw up other charts for other times, and repeat this process many times, rubbing out and re-drawing, probably all of the charts in some way or other, to achieve the final goal of sensible continuity. Each time a chart is readjusted the changes needed will become smaller and smaller, until finally the forecaster is happy that they have a full and consistent sequence. This is the forecaster's equivalent of 4D-Var. Of course in 4D-Var there is the additional constraint of full vertical consistency also, though in the forecaster world soundings and imagery may indeed be contributing in an analogous way.
Meanwhile the Ensemble of Data Assimilations (EDA) could be thought of as being equivalent to the above process, but activated several times producing slightly different smooth and continuous but equally probable sequences. Where there are lots of observations the several sequences might end up almost the same, but where data is sparse one could end up with much more variability between sequences. And where there is large dynamic instability (e.g. a developing frontal wave rather than the centre of an anticyclone) this 'spread' might be even greater. In the real EDA this is essentially also what happens; more spread is found amongst members in data sparse areas and when there is larger innate uncertainty in actual atmospheric developments.
Additional Sources of Information
(Note: In older material there may be references to issues that have subsequently been addressed)
- Read more on 4D-Var.
- Read more on the use of EDA in 4D-Var.
- Read more on data assimilation and interpolation of data.
- Watch a webinar on Data assimilation and 4D-Var (30sec delay).