Last modified on Jan 14, 2025 10:56

Contributors: 

Issued by: S. Schimanke (SMHI)

Issued Date: 16/07/2024

Ref: C3S2_D360_Lot1.4.1.1_v2

Official reference number service contract: 2022/C3S2_360_Lot1_SMHI/SC1 

1. Executive Summary

This document gives an overview on the current understanding of strengths and weaknesses of CERRA and CERRA-Land for the available data (1 September 1984 – 30 June 2021) at the time of writing. 

The main conclusions: 

• CERRA vs. ERA5
  • CERRA outperforms ERA5 for 2m temperature (T2m) and 10m wind speed and direction (see Sections 2.1, 2.2, and 2.3).
  • The representation of large-scale features, e.g. geopotential at 500 hPa, in CERRA is similar to ERA5 (see Section 2.1).
  • Long-term trends/climate change signals agree in most areas between CERRA and ERA5 giving confidence in both products. Differences appear in areas with varying topography where it can be assumed that CERRA outperforms ERA5 due to a better resolved topography in the system (Section 2.6).
• CERRA vs. UERRA-HARMONIE
  • CERRA has better quality for T2m and 10m wind speed (Sections 2.2 and 2.3)
  • CERRA comes with a very much reduced spin-up after the analysis adding value to the short-range forecast time steps (Section 2.7)
  • CERRA is more homogeneous in time than UERRA-HARMONIE (Section 2.8)
• CERRA-Land vs. CERRA/ERA5/ERA5-Land/MESCAN-SURFEX
  • 24-h total accumulated precipitation analysis in CERRA-Land improves upon the precipitation forecast in CERRA (Section 3.1)
  • Extreme precipitation events in CERRA-Land improve upon MESCAN-SURFEX (Section 3.1)
  • Snow depth in CERRA-Land outperforms CERRA, MESCAN-SURFEX and ERA5-Land (Section 3.2)

Complementary results can be found in Ridal et al. (2024).

Identified weaknesses/shortcomings:

• For the CERRA-Land 24-h total accumulated precipitation (RR24) analysis, there is a reduction in the number of precipitation observations over the Eastern part (East of 50E) of the domain since 1999, except for a short period at the beginning of 2012, because of changes in the report times synchronised to 03 UTC instead of 06 UTC. CERRA-Land RR24 analysis therefore reverts to representing CERRA background forecast precipitation for this region (Section 3.1).
• CERRA: reduced observation usage during certain periods, e.g. SYNOP and buoy data (Section 2.5).

2. CERRA

2.1. Verification based on the HARMONIE package, CERRA vs ERA5

We have used the HARMONIE verification tool (Monitor, https://hirlam.github.io/Monitor/dev/) to compute statistics of CERRA and ERA5. Monitor uses a predefined list of observing stations which serves as the reference true state. Model data is interpolated to the observation locations and standard statistical metrics such as bias and standard deviations are computed. There are two station lists used, one with SYNOP stations and one with radiosondes for upper air verification.
Each year in the reanalysis period has been divided into four seasons: winter (December to February, DJF), spring (March to May, MAM), summer (June to August, JJA) and autumn (September to November, SON). The verification statistics are computed for each season separately, thus each data point in Figures 1-3 represents a verification of a 3-month period.

2.1.1. Surface pressure and geopotential

The regional model system (CERRA) uses ERA5 as lateral boundaries and thus it can be expected that the large-scale representation of CERRA is similar to that of ERA5. Figure 1 shows verification statistics for surface pressure and geopotential at 500hPa, both parameters represent the quality of large-scale features. The verification at analysis time 00 and 12 UTC, Figure 1 (a and b), shows the model fit to observations as the stations used for verification have been assimilated. In terms of standard deviation of the error (stde), CERRA and ERA5 are very similar. Statistics of geopotential show a gradual decrease of the error with time. Surface pressure has a similar trend until 2005 when the errors increase. This has not been fully investigated but since CERRA and ERA5 show the same behavior, this is most likely due to something with the verification system itself, e.g. erroneous pressure observations have been used in the verification. It does therefore not mean that the quality of CERRA or ERA5 is degraded from 2005 and onward. The main conclusion from Figure 1(a, b) is that CERRA and ERA5 have similar quality, as expected, for large-scale features.

For 12-h forecasts, ERA5 is better than CERRA for surface pressure and 500hPa geopotential (Figure 1 (c,d)). Global models are usually better at predicting large scale phenomena than limited area models. For some parameters, like upper air wind and temperature, CERRA has a better fit to observations than ERA5 at analysis time and a worse fit at forecast lead time 12h (not shown). It should be noted that CERRA has been tuned in such a way to provide a good analysis and 3-hour forecast, which serve as a first guess to the next analysis cycle.
CERRA has a higher horizontal resolution and has different model physics. In terms of surface parameters linked to small scale features CERRA has the potential to add value over ERA5.

Figure 1: Upper left, 1a: Surface pressure at analysis time 00 and 12 UTC verified against SYNOP observations. Upper right, 1b: 500 hPa geopotential at forecast lead time 0 verified against radiosonde observations. Lower left, 1c: Surface pressure at forecast lead time 12h verified against SYNOP observations. The forecasts are started from the analyses at 00 and 12 UTC. Lower right, 1d: 500hPa geopotential at forecast lead time 12h verified against radiosonde observations. CERRA is blue and ERA5 is red, thick lines are standard deviation of the error and thin lines show the mean error, or bias. 

2.1.2. 10m wind, 2m temperature, 2m relative humidity

Figure 2 shows the verification of wind speed and direction at 10 metres. In terms of standard deviation (stde), CERRA has a lower error than ERA5, and it is quite consistent over time. Observations of wind at 10 metres are not assimilated so the observations are in this case independent from the model. 2-metre temperature also has a consistently lower error than ERA5 (Figure 3a and Figure 3c). For 2-metre relative humidity, Figure 3 (b,d), CERRA and ERA5 are very similar. For some periods of time CERRA is slightly better than ERA5 but it is not consistently so. The improvements in 10m wind and 2m temperature are retained at forecast lead time 12h, Figure 2c, 2d and Figure 3c. 

Figure 2: 10-metre wind at analysis time (00 and 12 UTC) and 12h forecast (initialized at 00 and 12 UTC) verified against SYNOP observations. Upper left, 2a: 10m wind speed at analysis time. Upper right, 2b: wind direction at 0h. Lower left, 2c: 10m wind speed at 12 hour forecast lead time. Lower right, 2d: Wind direction at 12h forecast lead time. CERRA is blue and ERA5 is red, thick lines are standard deviation of the error and thin lines show the mean error, or bias. 

Figure 3: 2m temperature and 2m relative humidity at analysis time (00 and 12 UTC) and 12h forecast (initialized at 00 and 12 UTC) verified against SYNOP observations. Upper left, 2a: 2m temperature at 0h. Upper right, 2b: 2m relative humidity at analysis time. Lower left, 2c: 2m temperature at 12 hour forecast lead time. Lower right, 2d: 2m relative humidity at 12h forecast lead time. CERRA is blue and ERA5 is red, thick lines are standard deviation of the error and thin lines show the mean error, or bias.

2.2. Time series of bias and standard deviation of T2m at selected stations  

Figure 4 shows the time evolution of the bias (12-month moving mean) for the monthly mean of the 2m temperature analysis for ERA5, both versions of MESCAN-SURFEX and the CERRA production (note that CERRA-Land does not provide 2m temperature). For Sodankyla, CERRA and MESCAN-SURFEX-V2 (a corrected version of MESCAN-SURFEX where incorrect T2m observations were removed, which had previously resulted in a cold bias in certain locations) are relatively similar and improve upon ERA5 and MESCAN-SURFEX. For Minsk, we clearly see the problem discovered by the users for some areas in MESCAN-SURFEX, solved in MESCAN-SURFEX-V2; CERRA has a smaller standard deviation and bias than ERA5. Also, for Paris-Orly and Stockholm-Bromma, CERRA outperforms ERA5.

Figure 5 shows a comparison of the monthly mean 2m temperature analysis for two stations in the Alps, one used probably in all assimilation cycles (ERA5, UERRA-HARMONIE and CERRA) Chamonix and an independent one Besse (1520m). The comparison is done for the period 1985-2003 by taking the nearest grid point with a height correction by using standard atmospheric temperature lapse rate (-6.5°K/km). However, both stations are in valleys and the nearest grid point for the 3 systems ERA5, UERRA-HARMONIE and CERRA is significantly higher 2181m, 2191m, 2074m (resp.) for Besse and even higher for Chamonix 2334m, 1774m, 1841m. For Chamonix, the standard deviation varies from 1.02K for MESCAN-SURFEX up to 1.25K for CERRA, but for Besse the standard deviation is better for CERRA (0.96K) compared to MESCAN-SURFEX (1.14K) and significantly better compared to ERA5-Land (1.47K). 

Figure 6 shows the bias and standard deviation between the monthly mean 2m temperature from the analysis and the observations (as shown for specific stations in Fig. 5). The reanalysis monthly means are height corrected to the station altitude by using the temperature lapse rate (-6.5 K/km). The station observations come from those assimilated in CERRA, so they are not independent. CERRA has a smaller bias than ERA5 and a smaller standard deviation for the whole period, which was also seen before adding the height correction (not shown). CERRA does better than MESCAN-SURFEX-V2 in terms of standard deviation. CERRA does better than MESCAN-SURFEX overall but it appears that MESCAN-SURFEX does marginally better after 2005. This is attributed to MESCAN-SURFEX being tuned to be closer to observations - a decision that was altered for MESCAN-SURFEX-V2 and CERRA to avoid errors from low quality observations. CERRA having smaller biases and standard deviations than ERA5 in these statistics is a valid strength and not due to "over-tuning" since it is shown in Figure 3 that the 12-hour forecasts have smaller standard deviations than ERA5.  

Figure 4: Time series of 12 months moving mean of the monthly 2m temperature analysis bias and standard deviation for ERA5, MESCAN-SURFEX, MESCAN-SURFEX-V2 and CERRA for 4 cities.


Figure 5: Comparison of monthly mean 2m temperature analysis for Chamonix (1042m) and Besse (1520m). Chamonix observations are supposed to be used in all re-analysis systems and Besse observations not.

Figure 6: 12 month moving means of biases and standard deviations between monthly mean 2m temperature from the analysis and the observations (see text for calculation details).

2.3. Comparison with Swedish observations

In the following section, CERRA along with its predecessor UERRA-HARMONIE (Copernicus Climate Change Service, 2019), and the widely used global reanalysis ERA5 (Hersbach et al., 2020), are evaluated with available sub-daily observational data from Swedish meteorological stations during the 30-year period 1989 – 2018. We aim to give a thorough showcase of three parameters: 2 metre temperature (T2m), 10 metre wind speed (WS10), and 2 metre relative humidity (RH2m). It should be mentioned that in CERRA, T2m and RH2m are parameters where observations are assimilated, whereas WS10 is only available from the forecast model and is therefore an example of a parameter independent from the station series. An inquiry on how well each model compare to observations on a spatial scale over Sweden is provided (Section 2.3.1) as well as within the seasonal cycle (Section 2.3.2). A comparison to station data along the Swedish coastline is provided in Section 2.3.3.

Sub-daily observations are obtained from SMHI's digitized observation database MORA (Meteorologisk Observationsdatabas för Realtid och Arkiv) (SMHI, 2023). These are selected on the basis of their accessibility. While limited to Sweden, we believe the dataset can provide a good overview of how the models perform in different physical features considering the regional diversity of the country. CERRA and UERRA-HARMONIE provide analysis every 3 and 6 hours, respectively. Forecast and analysis are therefore combined to match the hourly resolution of automatic stations. In the absence of RH2m as a ready-available parameter in ERA5, it has been calculated using the T2m and 2 metre dew point temperature following the Magnus Formula with revised Magnus coefficients as recommended by Alduchov and Ekedge (1996): 

Where: T = Air temperature, Dp = Dew point temperature, 243.04°C & 17.625 = revised Magnus coefficients

Reanalysis offer advantages over in-situ measurements by providing records of meteorological variables consistent in both time and space. However, a general limitation is their inability to recreate local weather phenomena and small-scale spatial patterns often found in complex topography such as coastal or mountainous regions considering that each grid point value describes the state of the entire grid box. In CERRA, an area of potential added value compared to other reanalysis models lie in the finer spacing of grid points in the model, having a horizontal resolution of 5.5 km, opposed to 11 km for UERRA-HARMONIE and 31 km for ERA5. As such, there is potential for CERRA to better capture the features described above. Other improvements may also come from the increased amount of data included in the assimilation procedure or differences in the model physics. 

2.3.1. Spatial variability

Figure 7 - Figure 9 display the RMSE for all stations, derived using all observed data series computed against their respective closest grid point data series from all three models. A height correction is not included in these analyses. For T2m (Figure 7), the RMSE of CERRA is at its lowest in the southern part of Sweden, at ca. 1 °C. In the middle to northern interior the RMSE have larger values with respect to the higher the elevation, around 2 – 3 °C and some outliers above 3 °C. When compared to ERA5, CERRA demonstrates ca. 0.2 - 0.5 °C lower RMSE especially noticeable within coastal sectors and the Scandinavian Mountains (inland above ca. 60°N, adjoining Norway) but with differences also being observed in the more homogeneous terrain of the south. In UERRA-HARMONIE, more stations situated in the middle to south are shaded orange, representing a RMSE around 2 °C, while in the north there are many more stations with a RMSE around 4 – 5 °C, meaning that both CERRA and ERA5 outperforms UERRA-HARMONIE for the parameter T2m. It is worth mentioning that for all the models there are a few outlier stations in the mountainous regions bordering Norway, that can be seen in a darker purple shade, meaning that these stations have a high RMSE for all models.

Figure 7: 2m temperature for the period 1989 - 2018. Climatology of observations (left) and RMSE computed for CERRA, ERA5 and UERRA-HARMONIE.

For WS10 (Figure 8), the difference between models can be somewhat harder to discern at first glance. However, while the majority of stations for all three models display RMSE values between ca. 1 – 3 m/s, CERRA does overall have a more yellow tone, signaling values in the lower part of this range. This is true countrywide, with the exception of a few outlier stations at higher terrain where all models exhibit a poor performance. Comparing UERRA-HARMONIE with ERA5 we find comparable values along the coastline and at higher elevation, while at middle - southern regions ERA5 generally tend to outperform UERRA-HARMONIE.

Based on the information in Figure 9, CERRA is outperformed by ERA5 in the representation of RH2m, with the exception of stations within the coastal vicinity where both models exhibit values at ca. 7 - 9 %. For ERA5, a paler shade of yellow stretches through the inland, which corresponds to more consistent values at around 7 %, i.e., ca. 1 - 2 % lower than CERRA. In the highly elevated inland, ERA5 exhibit values only slightly larger than coastal stations, whereas CERRA has a darker shade of orange, which reflects values at ca. 10 - 12 %. Both CERRA and ERA5 outperforms UERRA-HARMONIE, which tend to show RMSE > 10 % for a majority of the stations and even more pronounced values at ca. 13 - 17 % within the middle - northern inland.

Figure 8: 10m wind speed for the period 1989 - 2018. Climatology of observations (left) and RMSE compared to observations for CERRA, ERA5 and UERRA-HARMONIE.

Figure 9: 2m relative humidity for the period 1989 - 2018. Climatology of observations (left) and RMSE compared to observations for CERRA, ERA5 and UERRA-HARMONIE.

2.3.2. Seasonal variability

In Figure 10 - Figure 12, we examine how the performance of CERRA (blue), ERA5 (orange) and UERRA-HARMONIE (green) vary with the seasonal cycle. Statistical metrics RMSE, bias and Pearson's correlation coefficient are computed for all Swedish stations to see an average score for each month of the year during the examined period. Figure 10 displays the three metrics for T2m. For RMSE (left), CERRA has on average a lower score during August until October and then higher for November until February. The other models follow the same pattern, but with ca. 0.2 - 0.7 °C higher RMSE, and with UERRA-HARMONIE recording a more pronounced inclination which indicates a larger error difference from month to month whereas the other models follow a more even difference throughout the year. In terms of bias (middle), CERRA generally has a slight underestimation of the temperature ranging between -0.3 to 0.1 °C. During the summer months the bias is near the 0 °C mark, while in autumn and winter it is considered to be around ca. -0.1 to -0.2°C, and in spring the underestimation peaks at ca. -0.3°C. ERA5 is instead overestimating the temperature with ca. -0.1 to 0.3 °C, with lower bias in April - July than CERRA, but overestimating the temperature in the autumn and winter more so than CERRA. UERRA-HARMONIE has a relatively small bias of ca. 0.1 °C in July and August, but is however underestimating the temperature much more than CERRA throughout the rest of the year. Concerning correlation between stations and models, all follow the same pattern, with higher correlation during autumn, winter and spring and lowest during the summer (Figure 10, right). The correlation varies between 0.92 to 0.97, and it can be observed that CERRA has the highest correlation with around 0.01 and 0.03 higher score than ERA5 and UERRA-HARMONIE, respectively.

Figure 10: T2m monthly evaluation scores: RMSE (left), bias (middle) and correlation (right), averaged for all stations and the period 1989 – 2018. The RMSE part of the figure is a copy of Figure 19 from Ridal et al. (2024).

In Figure 11, the same metrics are shown for WS10. For RMSE (Figure 11, left), CERRA has on average a smaller error throughout the year than ERA5 and UERRA-HARMONIE. In general, all the models have lower errors during summer but higher during the rest of the year with a peak in winter. For bias (middle), CERRA is close to the 0 m/s mark for the winter and autumn months but has a negative bias during the spring and summer, reaching as low as ca. -0.4 m/s in May. In contrast, ERA5 and UERRA-HARMONIE are not as biased in spring and summer but instead overestimate the wind speed in the other seasons up to ca. 0.3 and 0.4 m/s. The average correlation for the three models follows a close pattern but with CERRA having the highest correlation. In the summer months the correlation is the lowest for all of the models, and is recorded to be around 0.70 – 0.75, and higher in the winter, spring and autumn, around 0.75 to 0.80.

Figure 11: WS10 monthly evaluation scores: RMSE (left), bias (middle) and correlation (right), averaged for all stations and the period 1989 – 2018. The RMSE part of the figure is a copy of Figure 19 from Ridal et al. (2024). 

For RH2m, all the models have higher RMSE during March to July and then lower for the rest of the year (Figure 12). While it is shown that ERA5 has an overall lower RMSE averaged over the whole year, in the months August until November CERRA has approximately the same error as ERA5. The bias (Figure 12 middle) ERA5 is more consistent throughout all the months of the year with a bias that is slightly below 0 and performs better than CERRA and UERRA-HARMONIE which overestimate RH2m in the first half of the year. From July to December, the bias is comparable with ERA5. The correlation (right) is generally the strongest in spring to autumn, models ranging between ca. 0.65 to 0.85, and weakest in winter with ca. 0.5 to 0.75. ERA5 has the overall highest correlation, but CERRA is closely following ERA5's pattern. When comparing CERRA and ERA5 to UERRA-HARMONIE, there is a distinct difference where the former models show between 0.1 to 0.2 higher correlation consistently throughout the year.

Figure 12: RH2m monthly evaluation scores: RMSE (left), bias (middle) and correlation (right), averaged for all stations and the period 1989 – 2018. The RMSE part of the Figure is a copy of Figure 19 from Ridal et al. (2024).

Here, we have validated CERRA, ERA5 and UERRA-HARMONIE against all available Swedish meteorological observations. For parameters T2m and WS10 we find that CERRA adds clear value to UERRA-HARMONIE and ERA5 with improved statistical scores, demonstrating the benefits of the higher resolution in the regional reanalysis. The models showed rather high evaluation scores for T2m, which can be anticipated with temperature being a parameter more constant in time and space, compared to say wind speed, which would have a higher dependency on local topography and surface roughness. Also, observations of T2m are assimilated in the model, so one would expect better agreement than when compared with WS10. In the case of RH2m, with the exception of coastal regions and the autumn months (averaged over all stations), the distribution of the statistical indices in CERRA are found at slightly lower scores when set side by side with ERA5. This is also consistent with the results from running the Harmonie verification tool (see Section 2.1) where findings often showed a slightly larger bias for CERRAs RH2m with values retained at forecast lead time 12h compared to SYNOP station data. However, the comparison with UERRA-HARMONIE shows that significant improvements have been made in CERRA, as it is outperforming UERRA-HARMONIE in all categories. 

2.3.3. Comparison to station data along the Swedish coastline

We validated CERRA, UERRA-HARMONIE and ERA5 also especially against more than 30 Swedish coastal stations. The observations come from automatic stations and here we use hourly data for the period 1996 – 2018 (last complete year for UERRA-HARMONIE). Investigations include 2m temperature and 10m wind speed.
Main conclusions:

• CERRA compares best to observations in terms of RMSE and correlation both for 10m wind speed as well as 2m temperature
• UERRA-HARMONIE outperforms ERA5 for 10m wind speed and has about the same quality for 2m temperature

Table 1: Statistics for wind speed at 10m. Statistics are based on hourly data for the period 1996 - 2018. Observations are taken from more than 30 automatic stations along the Swedish coastline. For reanalysis products, the closest grid point was taken.

Table 2: Statistics for 2m temperature. Statistics are based on hourly data for the period 1996 - 2018. Observations are taken from more than 30 automatic stations along the Swedish coastline. For reanalysis products, the closest grid point was taken. 

2.4. Surface shortwave radiation

Figure 13 shows monthly time series of the bias (upper), RMSE (middle) and the mean (lower) of the surface solar radiation downwards over France for ERA5 and CERRA. Observations of surface solar radiation from 250 ground stations have been used to calculate the bias and the RMSE. Figure 13 shows that the bias and RMSE of the surface solar radiation downwards during summer is smaller for ERA5 rather than for CERRA whereas during fall and winter the bias is smaller for CERRA rather than for ERA5 (smaller bias and RMSE is better). 


Figure 13: Time series of the bias (upper), RMSE (middle) and mean value (lower) of the surface solar radiation downwards (W/m²) at monthly time scale over France.

2.5. Observation usage

2.5.1. Number of assimilated observations

Figure 14 shows the number of daily mean observations assimilated in CERRA for two streams: cerra_2 corresponding to the beginning of the reanalysis period in 1985 and CERRA-NRT for the near real time integration in 2019. Conventional observations as well as satellite radiances (SATEM) are indicated. The total amount of observations corresponds to the black line, and there is a large increase of this number over time, from 50,000 daily observations in 1985 to 350,000 in 2019. The main reason is the increase in the number of aircraft (red line) and satellite radiances (orange line) measurements entering the system.

Figure 14: Number of daily observations per type of observation in 1985 (left panel) and 2019 (right panel).

We have then looked at the time-evolution of the number of observations assimilated into the system as shown in Figure 15. For that purpose, the number of observations in 1985, 1991, 1998, 2005, 2012 and 2019 for the month of January were considered. As expected, there is an almost constant amount of TEMP over time. The same for DRIBU, PILOT and SYNOP. The main changes concern the SATOB with a small increase after 2000, the AIREP, corresponding to a large increase in air traffic and the possibility to measure more atmospheric quantities, and finally in the SATEM satellite radiances measurement, which represent about half of the daily observations in the CERRA analysis system from 2012 and onwards. 
The decrease of PILOT in time seems realistic since it was a developed activity in the 80's but nowadays there are more wind profile observations available, e.g. from radar (very high frequency wind profilers) or satellite measurements (atmospheric motion vectors, scatterometers, microwave sounders, etc.) and only a few balloons are launched.

Figure 15: Mean number of daily observations in January used in the CERRA reanalysis system.

A known problem with the observation usage is that for approximately 3 months in 2019, and from 1st of April 2020 and until the end of March 2021, very few ocean buoy observations were included in the data assimilation in CERRA and CERRA-EDA.  
Moreover, a drop of SYNOP observations occurred on 1^st of October 2020 (Figure 16) after changes in the BUFR-encoding.

Figure 16: Number of SYNOP stations for the period Jul 2020 to Jul 2021.

In addition, there are a small number of less serious deviations from the nominal usage of observations and other input data in CERRA and/or CERRA-EDA. These deviations are:

• The CERRA production was produced with a slightly degraded B-matrix for approximately 2 months for the beginning of 2019. The impact is considered to be negligible. CERRA-EDA was not affected.
• January to May 2019 of CERRA-EDA was produced without the atmospheric motion vector data.

2.5.2. Feedback statistics

An optimal interpolation (OI) of the screen level variables T2m and RH2m, and the surface variable snow water equivalent (SWE) are performed in the CERRA system online every 3 hours. The first guess and analysis departure statistics were collected and are presented for T2m and RH2m in Figure 17 and Figure 18, respectively.
The mean number of observations per analysis increases threefold from approximately 1500 to 4500, from the mid-1980s to the late 2010s. Looking closely, it seems that some trends in the departure statistics can be seen, but that these are small compared to the scale of the standard deviation of the departures.
The standard deviation of the departures gives an indication of the accuracy of the first guess. The standard deviation of T2m is approximately 1-2K and of RH2m is approximately 10%. The analyses have smaller standard deviations than the backgrounds because they are incremented towards the observations, but a standard deviation still remains, presumably related to the representation differences between the model field at 5.5km grid spacing and the real temperature/humidity fields containing small scale fluctuations.

Figure 17: CERRA T2m monthly mean first guess departure (equivalently fg_depar, Obs-minus-Firstguess, OmF), analysis departure (equivalently an_depar, Obs-minus-Analysis, OmA) and obs numbers. The shaded regions are from +1 to -1 standard deviations from the means of fg_depar and an_depar respectively, to represent the monthly distribution of departure values.

Figure 18: CERRA RH2m monthly mean first guess departure (equivalently fg_depar, Obs-minus-Firstguess, OmF) analysis departure (equivalently an_depar, Obs-minus-Analysis, OmA), and obs numbers. The shaded regions are from +1 to -1 standard deviations from the means of fg_depar and an_depar respectively, to represent the monthly distribution of departure values.

In the surface analysis done in the 3DVAR cycle, the number of observations in CERRA has been significantly increased compared to UERRA (Figure 19), however this increase of observations depending on the area is not homogenous in time and space especially for Eastern Europe (Figure 19 bottom). The first guess of the T2m of the CERRA system is also significantly improved thanks to the increase of the horizontal resolution (11km for UERRA-HARMONIE and 5.5 km for MESCAN-SURFEX-V2 and CERRA) and to a shorter forecast length (6h for UERRA-HARMONIE and 3h for CERRA).

Figure 19: CERRA (blue) and MESCAN-SURFEX-V2 (red) T2m monthly mean first guess departure (equivalently fg_depar, Obs-minus-Firstguess, OmF) and analysis departure (equivalently an_depar, Obs-minus-Analysis, OmA). T2m observation numbers used in the analysis for MESCAN-SURFEX-V2 (orange) and CERRA (cyan). Top: for the entire domain; Bottom: Eastern Europe. 

2.6. Long-term trends

As described in Ridal et al. (2024), to verify that long-term trends as well as climate variability are well represented in CERRA, we compare long-term mean differences (2003–2020 minus 1985–2002) and annual variability of CERRA with ERA5. Diagnostics that include comparisons between CERRA, ERA5 and observations are shown in previous sections.

2.6.1. Temperature

Figure 7 in Ridal et al. (2024) shows that for both reanalyses, the temperature increase is more pronounced to the northeast of Europe. In general, the patterns and the amplitude agree very well between CERRA and ERA5 giving confidence to both datasets. In case of differences in the signals these are often present over areas with varying topography, e.g. the Alps. Here, it can be assumed that CERRA outperforms ERA5 due to a better resolved topography coming from the finer horizontal resolution. 

CERRA time series of annual mean T2m show high correlation with ERA5 and observations. Figure 20 shows the time series for Norrköping (Sweden) and examples for other European cities can be found in Ridal et al. (2024). So, beside reliable climate trends even the year-to-year variability is realistic in CERRA.

It should be noted that it is not expected that CERRA outperforms ERA5 when areas or time scales are aggregated. CERRA's added value comes with the higher resolution and therefore added value is expected for small regions and on shorter time scales. As seen in Section 2.3, the added value of CERRA shrinks or disappears when simple statistics as mean values are considered. Hence, even when moving from hourly resolution to monthly means (as below) the added value of CERRA becomes smaller. The figures in this section document in the first instance that climate trends and climate variability are well represented in CERRA. 

Figure 20: Annual mean T2m computed from observations, CERRA and ERA5 reanalysis.

Differences in the temperature signal over the Baltic Sea (as noticed in Figure 7 of Ridal et al. (2024)) are most pronounced during winter (DJF, Figure 21). For spring and autumn, seasonal means (Figure 22) are almost identical over time. The stronger temperature signal over the Bothnian Bay and the Gulf of Finland might be related to the used SST and sea-ice concentration. Whereas ERA5 is constantly colder during winter months until 1999, it becomes constantly warmer from 2000 onwards. This change coincides with the switch between different regional ocean reanalysis used as input datasets for the Baltic Sea, one based on RCO (Liu et al. 2013) and the other on NEMO (Axell, 2021) .

Figure 21: Climate change 2m temperature signal during winter (DJF). Differences are shown for 2003-2020 minus 1985-2002 for CERRA (upper left) and ERA5 (upper right). Differences between CERRA and ERA5 signals are shown in the bottom figure.

Figure 22: Temperature series of seasonal means average over the Bothnian Bay both for CERRA and ERA5.

2.6.2. Wind speed

Similar to T2m, we compare for the wind speed at 10m the climate change signal in CERRA to the signal of ERA5 (Figure 23). We see that the mean changes are in general rather small with a tendency to slightly stronger winds in southern Europe and a slight weakening in central and northern Europe. This overall pattern is identical for CERRA and ERA5 providing confidence for both datasets. 

Figure 23: Climate change 10m wind speed signal. Differences are shown for 2003-2020 as percentage of 1985-2002 for CERRA (upper left) and ERA5 (upper right). Differences between CERRA and ERA5 signals are shown in the bottom figure. 

2.7. Improved short-range forecasts compared to UERRA-HARMONIE

As documented in the user guide (documentation page) and on confluence pages, UERRA-HARMONIE wind speed suffers from a bad initialization of turbulent kinetic energy. The first forecast hours are affected. The issue was resolved for CERRA. An example is given in the Figure 24. It shows jumps of wind speed after the analyses at 00, 06, 12 and 18 UTC for UERRA-HARMONIE. This behavior disappears for CERRA having a smooth evolution of wind speed. 

Figure 24: Time series of 10m wind speed during the storm Gudrun at the location Växjö.

In addition, UERRA-HARMONIE has a spin-up problem affecting precipitation in the first forecast hours where total precipitation is underestimated. For CERRA, improved initialisation of cloud properties resolved this issue to a large degree (Fig. 25). 

Figure 25: Averaged total precipitation for the entire domain for different forecast lengths.

2.8. Forecast skill

As no model is perfect, separate analysis errors and forecast errors exist. When comparing a forecast to an analysis, both errors naturally affect the result. However, comparing two successive forecasts valid for the same time isolates the forecast error, and makes it possible to solely focus on investigating the forecasting skill. If the differences between two separate forecasts, valid at the same valid time, change systematically with time, the dataset is temporally inconsistent. Here, we consider differences between the 6-hours forecast and the 30-h forecast, which are both valid for the same time. For instance, the 30-h forecast initialised on 1 January at 12 UTC and the 6-h forecast initialised on 2 January at 12 UTC are both valid on 2 January at 18 UTC.
It should be noted that the forecast skill includes only little information about the quality of the data itself since there is no comparison to observations. However, it informs about the homogeneity of the data series over time, which is important for reanalysis datasets.

Figure 26: Forecast skill based on the geopotential height at 500 hPa for winter months. Monthly mean difference and standard deviation are shown based on forecast initialized at 12UTC. Differences and standard deviation are taken between the 6-h forecast and the 30-h forecast valid for the same time. 

Figure 26 shows that CERRA is a homogeneous dataset over the entire period. There are no significant jumps in the standard deviation (red line). The gentle negative trend of the standard deviation can be explained by the increasing number of available observations over time - especially aircraft and satellite data, which improves the analyses over time. As a consequence, the forecasts become more accurate as visible by the negative trend. Moreover, there is only a small negative systematic difference between the forecasts in the geopotential (Figure 26, blue line) indicating that the model is not drifting over the forecast window. The small systematic differences tell us that no severe errors are present in the forecast model, which might accumulate over time. Altogether, Figure 27 gives confidence that the forecast model works as expected and that CERRA is homogenous in time.

The homogeneity in time is clearly an improvement compared to UERRA-HARMONIE. For example, in the corresponding UERRA-HARMONIE time series (Figure 27), a strong reduction of the standard deviation occurs at the transition from ERA40 to ERA-interim as lateral boundary data indicating that the quality of UERRA-HARMONIE is affected by the switch of the lateral boundary conditions. 

Figure 27: Same as Figure 23 but for UERRA-HARMONIE. 

2.9. Transition between streams

CERRA was produced with 13 independent streams. Each stream was spun-up for at least one year before the production of the final product started. Whereas the response time of the atmosphere is rather short, soil parameters need significantly more time to spin-up. In general, we cannot detect any spin-up effect between the streams for atmospheric parameters (not shown), but the transition between streams is still visible for soil parameters and here mainly for liquid volumetric soil moisture. 

c)
d)
Figure 28: Time series of liquid volumetric soil moisture for all levels in CERRA for all decades covered in CERRA: a) 2010-2020, b) 2000-2009, c) 1990-1999, d) Sep 1984 - 1989. The series show data for a grid box in Spain, an area with rather long spin-up times compared to other regions (not shown).

Figure 29: Same as Figure 28 but zoomed into the transition in August-September 2002.

Conclusion based on Figure 28 and Figure 29:

• For most of the seams, there is a jump in the deepest layer for the liquid volumetric soil moisture indicating that a spin-up of one year is too short for deep soil moisture. 
• In the middle layer, transitions between the streams are already hardly noticeable.
• The top layer is that noisy - reacting very quickly - that no signs of the seams can be notified.

It is well known that the deep soil needs a long time to get in balance with the forcing. That's why CERRA-Land is produced with one stream only, which was spun-up for ten years before production started. Overall, atmospheric variables and the top soil level are not affected by the transition of the streams. They are seamless. Only the deep soil levels are affected in CERRA. The deep soil variables of the CERRA reanalysis are not published in the CDS and users interested in them must use the CERRA-Land products.

3. CERRA-Land

3.1. Daily precipitation in CERRA-Land

Daily total accumulated precipitation (RR24) analysis in CERRA-Land improves upon the precipitation forecasts in CERRA in that it is an optimal interpolation (OI) analysis between rain gauge observations and a background generated as a sum of eight 3-hour total precipitation forecasts of CERRA. The advantages are that the OI analysis more accurately represents the historical precipitation, and also corrects for model biases. The disadvantage is that due to a changing observation network, trends might not be accurately represented, because when observations are not present, model biases might modify the climatology (this has been noticed over the Eastern part of the domain). It is expected that the users would consider ERA5, CERRA and CERRA-Land precipitation if their study was sensitive to this.

Firstly, a case study of an extreme event is presented to show a concrete example of improvements compared to the predecessor MESCAN-SURFEX, which also performed an optimal interpolation. The RR24 observation numbers have approximately doubled in CERRA-Land compared to MESCAN-SURFEX, as seen in Figure 30, largely thanks to the European Climate Assessment and Dataset (https://www.ecad.eu/) project, but also to provided data from France, Sweden, Norway, Finland, Iceland and Denmark.

Figure 30: Obs-numbers by source in MESCAN-SURFEX (left) and CERRA-Land (right). Note: CE_v5_04feb21-Estonia is the technical name for the final version of input observations before quality control.

Figure 31 shows the observed values in UERRA-Harmonie and CERRA for a case study of an extreme event in the south of France on 13 Nov 1999. With the added French observations, the extreme event is better represented. It was found that the quality control removed the maximum observed value in MESCAN-SURFEX, but kept it in CERRA-Land probably because there are more similar large values of precipitation, not available in MESCAN-SURFEX.

Figure 31: Observed values of the extreme event of 12 – 13 Nov 1999 in the south of France: MESCAN-SURFEX (left panel) and CERRA-Land (right panel).

The increase in horizontal resolution of CERRA compared to UERRA-HARMONIE allows for a more realistic estimation of localized extremes. Figure 32 shows the RR24 background forecast for MESCAN-SURFEX (based on UERRA-HARMONIE) and CERRA-Land (based on CERRA) respectively and that indeed, the peak value of precipitation is larger in CERRA. This, however, does not guarantee a more accurate forecast since the local extreme might not be forecasted in the correct location, which was found to be the case in this study.

Figure 32: The daily total accumulated precipitation forecast valid at 06 UTC on 13 Nov 1999 from UERRA first guess (left panel) and from CERRA first guess (right panel).

Figure 33 shows the CERRA-Land RR24 analysis and its difference with MESCAN-SURFEX. The CERRA-Land analysis finally has a better representation of the peak value and is more localized than that of MESCAN-SURFEX.

Figure 33: CERRA-Land analysis of the 24 h accumulated precipitation valid at 06 UTC 13 Nov 1999 for the flash flood event in the South of France.

The CERRA-Land RR24 analysis improves the representation of the water cycle. However, users need to be warned about changing distribution of surface station observations which can affect trends. This is demonstrated as follows over Russia where the surface observation network changes in time. There are 2 major changes, the first on 27 Nov 1999 as shown in Figure 34. This change was found to be associated with a change in the report times of the observations. Precipitation accumulations in Eastern regions were changed to be in 12 h accumulations reported at 03 UTC and 15 UTC which does not match the MESCAN configuration of 06 UTC to 06 UTC. This was discovered relatively late; if it had been discovered earlier we would have considered including 24h accumulations at 03 UTC to 03 UTC counting them as 06 UTC - 06 UTC, accepting an error from the 3h time difference. This is effectively what we decided for some Icelandic observations that are accumulated between 09 UTC to 09 UTC.

Figure 34: 24h accumulated precipitation sources on the 26 and 27 Nov 1999, which shows the first major change in reporting of Russian obs.

To assess the impact of the change in observation distribution, a so-called Russian domain was defined, marked in Figure 34, where the time series of observation numbers was produced, as shown in Figure 35. The change in the spatial distribution (shown in Figure 34) can be clearly seen in the drop in observation number (on 26 November 1999) in the time series (Figure 35), which also corresponds to an increase in the annual mean of the monthly accumulated and spatially averaged precipitation. This is because the first guess in CERRA overestimates precipitation relative to the observations. The observations are not guaranteed to be free of biases, notably during the cold winter conditions and effects such as the underestimation of snow.

Figure 35: Daily time series of observation numbers over a so-called Russian domain (marked in Fig. 33) shown in black and the annual mean of the monthly spatially averaged means of accumulated precipitation in blue.

Another change in the observation distribution was around the autumn of 2012 (Figure 35). A short peak in observation numbers (31 August 2012) corresponds to an increase in the density of observation east of 50E. Their decrease in number corresponds to observation being discarded east of approximately 50E. Therefore, users need to be aware of the change in spatial distributions of the surface observation networks to avoid misleading evaluation of trends especially east of 50E. This will also have a downstream impact on other surface parameters, produced by CERRA-Land, such as soil water content, snow and surface fluxes.

Different scores were calculated such as the bias, the Root-Mean Square Error (RMSE), the Heidke Skill Score (HSS), the probability of detection (POD) and the false alarm rate (FAR). The HSS gives information on analysis accuracy, the POD measures the probability that an observed event is well forecasted by the system. POD is between 0 and 1 (1 for a perfect forecast). The FAR gives the probability that an expected event did not occur. FAR is between 0 and 1 (0 for a perfect forecast). The scores were calculated over France with a daily independent network of rain gauges stations (~2000 stations/day) during 10 years (1990-2000).

Table 3: Bias, RMSE and temporal correlation (CORR) between in-situ precipitation measurements and closest grid point of the CERRA-Land daily precipitation analysis, the daily MESCAN-SURFEX precipitation analysis, the first guess based on CERRA, and the daily ERA5 surface precipitation. 

The Table 3 and Figure 36 show that the CERRA-Land precipitation analysis has better scores than MESCAN-SURFEX precipitation analysis. It improves the representation of precipitation over France. Furthermore, the daily surface precipitation of CERRA has a smaller bias and RMSE than those from ERA5.

Figure 36: Heidke Skill Score (HSS), False alarm rate (FAR) and the probability of detection (POD) were calculated for the CERRA-Land precipitation analysis (CERRA_an in the legend), the MESCAN-SURFEX precipitation analysis (UERRA_an) and, the CERRA precipitation (CERRA_bg) and ERA5 precipitation (ERA5). For POD and HSS the perfect score is 1 whereas it is 0 for FAR.

3.2. Snow depth evaluation

Figure 37 shows the added value of CERRA-Land versus CERRA for the monthly mean evolution of the snow depth (SD) over the French Alps. The SAFRAN–ISBA–MODCOU (Le Moigne et al., 2020) (hereafter referred to as SAFRAN) hydrometeorological model system (black line) can be considered as a reference (the truth), nevertheless, for high mountains it is probably less relevant. The snow depth over the French Alps is significantly improved by the CERRA-Land production versus the CERRA one especially for altitude below 2500m. Above 2500m, CERRA-Land has less snow depth than CERRA, in better agreement with SAFRAN. This improvement is due to the use of the precipitation analysis done with MESCAN (used to drive the SURFEX-Land model) and the use of a more sophisticated soil and snow scheme in CERRA-Land.

Figure 37: Time series of the monthly mean snow depth modelled over the French Alps by SAFRAN (black), CERRA (purple) and CERRA-Land (red) for different elevation range, 500-1500 m (upper), 1500-2500m (middle), and higher than 2500m (lower). The black line is considered as the reference. Bias and Rms are computed between January 1985 and December 2013.

Figure 38 shows also several time series of snow depth monthly mean from several surface re-analysis such as MESCAN-SURFEX (blue line), ERA5-Land (green line) and CERRA-Land (red line) and the operational model SAFRAN (black line) used at Météo-France. Both regional surface reanalysis MESCAN-SURFEX and CERRA-Land outperform significantly ERA5-Land in terms of bias and RMS error. 
CERRA-Land improves also MESCAN-SURFEX for the 3 elevation categories especially for elevation greater than 2500m. The resolution of the background at 5.5km in CERRA compared to the 11km one interpolated on a 5.5km grid for MESCAN-SURFEX is one of the reasons for and the other one, at least for the French-Alps, is the quality of the precipitation analysis thanks to more observations used for the analysis over France.

Figure 38: Time series of the monthly mean snow depth modelled over the French Alps by SIM (black), MESCAN-SURFEX (blue), ERA5-Land (green) and CERRA-Land (red) for different elevation range, 500-1500 m (upper), 1500-2500m (middle), and higher than 2500m (lower). The black line is considered as the reference. Bias and Rms are computed between January 1985 and December 2013.

Unfortunately, the number of snow depth observations above 2500m is very small, with some uncertainties, and some questions about the spatial representativeness of the observations. However, Figure 39 shows three "NIVOSE" stations (not used in CERRA and in CERRA-Land), the nearest grid point and the mean of the 3 nearest grid points from the 3 surface reanalyses (ERA5-Land, MESCAN-SURFEX, CERRA-Land) have been extracted to do the time series comparison. "NIVOSE" is the name of the network of automatic high mountain weather stations created by Météo-France to give access to real-time weather data for hard-to-reach mountain areas. Figure 39 shows only the result with the nearest grid point (the conclusion is the same with the 4 nearest grid point). The CERRA-Land snow depth is in better agreement with the observations, it is particularly true for Aigleton with a significant improvement, except for winter 2010 and 2011. For two other sites located at around 2900m, CERRA-Land also outperforms MESCAN-SURFEX and ERA5-Land with less underestimation of snow depth.

Figure 39: Time series of the monthly mean snow depth modelled over the French Alps by MESCAN-SURFEX (blue), ERA5-Land (green) and CERRA-Land (red) for 3 NIVOSE stations: Aigleton, Les Ecrins and Plagne-Bellecote (black dot).

4. Peer-reviewed publications

A selection of peer-reviewed publications demonstrating the quality of the CERRA, CERRA-EDA and CERRA-Land components.

El-Said et al. (2021): A new temporally flow-dependent EDA estimating background errors in the new Copernicus European Regional Re-Analysis (CERRA), Earth and Space Science Open Archive [pp. 28], doi: 10.1002/essoar.10507207.1, https://doi.org/10.1002/essoar.10507207.1 

Wang and Randriamampianina (2021): The Impact of Assimilating Satellite Radiance Observations in the Copernicus European Regional Reanalysis (CERRA), Remote Sensing , Vol. 13, No. 3, https://doi.org/10.3390/rs13030426  

Spangehl, T., Borsche, M., Niermann, D., Kaspar, F., Schimanke, S., Brienen, S., Möller, T.,  Brast, M. (2023): Intercomparing the quality of recent reanalyses for offshore wind farm planning in Germany's exclusive economic zone of the North Sea, Advances in Science and Research , Vol. 20, p. 109-128, https://asr.copernicus.org/articles/20/109/2023/

Monteiro, D. and S. Morin (2023): Multi-decadal past winter temperature, precipitation and snow cover information over the European Alps using multiple datasets, EGUsphere , Vol. 2023, p. 1-62, https://egusphere.copernicus.org/preprints/2023/egusphere-2023-166/

François, H., Samacoïts, R., Bird, D.N. et al. Climate change exacerbates snow-water-energy challenges for European ski tourism. Nat. Clim. Chang. 13, 935–942 (2023). https://doi.org/10.1038/s41558-023-01759-5

Ridal, M., Eric Bazile, Patrick Le Moigne, Roger Randriamampianina, Semjon Schimanke, Ulf Andrae, Lars Berggren, Pierre Brousseau, Per Dahlgren, Lisette Edvinsson, Adam El-Said, Michael Glinton, Susanna Hagelin, Susanna Hopsch, Ludvig Isaksson, Paulo Medeiros, Esbjörn Olsson, Per Unden, Zheng Qi Wang (2024): CERRA, the Copernicus European Regional Reanalysis system, Quarterly Journal of the Royal Meteorological Society, 1-27. Available from: https://doi.org/10.1002/qj.4764

5. References

Oleg A. Alduchov and Robert E. Eskridge (1996): Improved magnus form approximation of saturation vapor pressure. Journal of Applied Meteorology and Climatology, 35(4):601 – 609.

Axell, Lars (2021):  Product User Manual for Baltic Sea Physical Reanalysis Product, BALTICSEA_REANALYSIS_PHY_003_011, report of the Copernicus Marine Service

Copernicus Climate Change Service, Climate Data Store (2019): Complete UERRA regional reanalysis for Europe from 1961 to 2019. Copernicus Climate Change Service (C3S) Climate Data Store (CDS). DOI: 10.24381/cds.dd7c6d66 

Hersbach et al. (2020): The ERA5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730):1999–2049.

Le Moigne, P., Besson, F., Martin, E., Boé, J., Boone, A., Decharme, B., Etchevers, P., Faroux, S., Habets, F., Lafaysse, M., Leroux, D., and Rousset-Regimbeau, F. (2020): The latest improvements with SURFEX v8.0 of the Safran-Isba-Modcou hydrometeorological model for France. Geosci. Model Dev.,13, 3925–3946, https ://doi.org/10.5194/gmd-13-3925-2020

Liu, Y., Meier, H. E. M., Axell, L. (2013): Reanalyzing temperature and salinity on decadal time scales using the ensemble optimal interpolation data assimilation method and a 3D ocean circulation model of the Baltic Sea, Journal of Geophysical Research: Oceans, number 10, volume 118, 5536 – 5554, https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1002/jgrc.20384

Ridal, M., Bazile, E., Le Moigne, P., Randriamampianina, R., Schimanke, S., Andrae, U., et al. (2024): CERRA, the Copernicus European Regional Reanalysis system. Quarterly Journal of the Royal Meteorological Society, 1–27. Available from: https://doi.org/10.1002/qj.4764

SMHI (2023); Ladda ner meteorologiska observationer, https://www.smhi.se/data/meteorologi/ladda-ner-meteorologiska-observationer