Last modified on May 28, 2024 11:14

1. Introduction

The Copernicus Arctic Regional Reanalysis service, developed under C3S contract 322 Lot 2, where we will refer to the code system and data set as CARRA, is targeted for a production of a 2.5 km grid resolution regional reanalysis for European Arctic regions for a 24-year period between 1997 and 2021.(later the dataset had been extended to cover the period from 1991 to present). The CARRA reanalysis is performed at ECMWF High Performance Computer (HPC) platform on two model domains covering Greenland and Iceland on the west side (CARRA-West), and Svalbard and Northern Scandinavia on the east (CARRA-East).

The CARRA reanalysis system builds on the basis of the operational HARMONIE-AROME (Bengtsson et al, 2017) model. HARMONIE-AROME is a kilometer-scale mesoscale non-hydrostatic Numerical Weather Prediction (NWP) system with primary purpose for operational weather forecasting. For this reanalysis, the system is adapted with necessary extensions and improvements to meet the requirement for a high quality and consistent data set covering a multi-decadal period.

The baseline version for the CARRA reanalysis system is the reference HARMONIE-AROME 40h1.1.1. For reanalysis, technical adaptations include introduction of interface to use a multiple, enhanced sources of input such as improved surface database describing physiographic features, reprocessed or local observation data sets, as well as retrieval and use of the latest available global reanalysis (ERA5) as lateral boundary. Also, an adaptation and enhancement in monitoring and verification was necessary. For configuration of CARRA reanalysis system, it was considered beneficial to inherit as much as possible the basic model system infrastructure from the operational setup for IGB (DMI and IMO) and AROME-Arctic (MET) to minimize the need for development and tuning. However, various differences exist between the setup for operational suites at the two domains, such as grid types (linear or cubic grids), surface physiographic database (PGD), assimilation options in data use and algorithm, forecast model options etc, so that homogenization efforts on configuration features have been necessary. In addition, it has been necessary, for the reanalysis project, to improve on description of critical surface processes involving permafrost regions in Greenland, Iceland and Svalbard, in order to secure the usefulness of the dataset for climate application.

For the re analysis, to account for use of ERA5 analysis as lateral boundary data and a need to cover multi-decade period with varying observation data coverage, some iterations have been necessary to derive suitable structure functions for the 3D Var data assimilation. A selection was made after intercomparing several sets of structure functions derived with ensemble data for both winter and summer seasons by different approaches, one with an ensemble of data assimilation through perturbation of observation ("EDA"), another with background vector perturbation ("BRAND"). For the control of large scale information in 3D-Var , two alternative schemes which have been developed in the HIRLAM community to exploit the information from the host model (here ERA5) inside the limited area domain, one through an explicit spectral nudging in the background term, ("LSMIX" i.e. large scale mixing), another through control of large scale error by a penalty term in the cost function, ("Jk"), are compared. From the input data side, enhanced physiographic data (PGD) including albedo over glacier and sea ice, high resolution ocean surface and sea ice data and enhanced in-situ surface observations, were to be included with anticipated benefit to the reanalysis products. In addition, a range of remote sensing data with radiances, radio occultation, scatterometer and polar AMV wind, is included, which is considered especially beneficial in view of the sparseness of the conventional observation network in the Arctic region.

A comprehensive system documentation deliverable report, Yang et al 2020, has been prepared in parallel with this report and provides more details on the extensions of system elements and input data sets. During the preparation stage, substantial efforts were spent to test and implement a variety of features as low hanging fruits, harvesting from many developments by the research communities behind the HARMONIE-AROME model. This report documents the testing and verification work performed in this connection. We highlight the tests with meteorological significance, to illustrate the background and reasoning for the decisions for choice of options among alternatives, and the justification for various releases of the CARRA development versions during its evolution toward final tagging.

2. Component evaluation and version evolution

In December 2017, the CARRA-alpha system, which was adapted from the candidate version for the "reference" HARMONIE-AROME 40h1.1.1 as maintained by the HIRLAM-C program, was tagged. From a verification comparison for four month-long periods representing different seasons in 2012 and 2017, the CARRA-alpha showed a clear advantage over that of ERA5 for fit to observations of main surface weather parameters at analysis time. The system was thus considered a CARRA baseline in terms of meteorological quality. Technical adaptation and other improvements were thereafter built individually on top of the baseline.

In October 2018, CARRA-beta.1, which collected all the component developments targeted by the reanalysis project both in terms of input data streams and changes in algorithm, source code and scripts, was tagged. The version was then validated extensively through five month-long periods (Table 2.1), representing different seasons selected from the 24 years reanalysis period, and for both domains. This validation shows overall positive results with the upgraded version both in comparison to ERA5 data and to CARRA-alpha. In December 2018, an updated version, CARRA-beta.2, was tagged with additional observation data streams activated and tuning implemented. CARRA-rc1 (release candidate 1), was tagged in March 2019, with which the 1-year warm-up runs for three production streams in CARRA were initiated. In June 2019, the update with final versions of the structure functions were made for both of the CARRA domains, becoming the last major update with significant meteorological impact. Among the last introduced revisions, apart from numerous technical updates, activation of last few local observation data streams, changes in structure functions and corrections in blacklisting data for several observation data streams are features with meteorological consequences. It is estimated that the discrepancies between the Carra versions rc1 and final version 1.0, although having impact on reanalysis results during the spin-up period, the memory of such is rather limited and do not compromise the quality of final reanalysis products in terms of consistency.

In October 2019, the final reanalysis system, CARRA.1, was tagged, implementing final adjustments of the output data stream and correction on blacklisting setup for satellite data.

Table2.1: The month-long episodes selected for CARRA evaluations 

From the assembly of CARRA.alpha (baseline versions) to that of CARRA.beta (a consolidated component development towards final reanalysis system), and to the final tagging of CARRA system, the CARRA team conducted an extensive amount of parallel experiments to evaluate and consolidate the features to be included in the final CARRA reanalysis system, including those on selection of grid types, harmonized model options, alternative options for large scale constraints in data assimilation, optimal derivation of climatological structure functions, and inclusion of different sources of observation data in reanalysis.

Table 2.2 is a concise summary of the numerical experiments carried out during the component evaluation phase and the main outcomes, which totals 114 months using part or all of the test episodes as shown in Table 2.1 These extensive evaluations form the scientific basis for the evolution of CARRA reanalysis system throughout the evolution of CARRA reanalysis system from alpha, beta to final releases.

Table 2.2: Numerical experiments conducted to test and evaluate CARRA components during the development phase toward CARRA beta and later releases. 

Detailed verification results for all the above numerical experimentation are collected in the internal CARRA monitoring wiki page¹ on, https://hirlam.org/trac/wiki/CARRA%20Reanalysis#EvolutionandstatusofCARRAreanalysissystemListofexperiments

In the following, we present some of the verifications to highlight the evaluations on individual components.

2.1. Selection of quadratic grid

In operational forecast setup, a cubic grid is used in the dynamical core of the DMI/IMO HARMONIE suite for Greenland-Iceland domain IGB (CARRA-West), whereas linear grid is used in the AROME-Arctic suite run at MET Norway (CARRA-East). To harmonize on grid type in CARRA for both of the reanalyses domains, a comparison with the reanalysis setup was made between model runs with linear, quadratic and cubic grids. For the CARRA-West domain, month long tests were made for a summer (June 2012) and a winter (January 2017) period between runs with cubic and quadratic grids, respectively. For the CARRA-East domain, runs with linear and quadratic grids are compared for four month-long periods representing different seasons. Figure 2.1 shows, as an example, a daily time series of model errors with quadratic or linear grids from 0 to 3-h forecast, in comparison to surface observations during month July 2012 for the CARRA-Ea st domain.

Figure 2.1: Daily averaged error in STD and BIAS validated against surface observations in CARRA-East domain for runs with quadratic (green) and linear (red) grids during July 2012, for Mean Sea Level Pressure, (upper panel), wind speed at 10 m, (mid panel) and temperature at 2 m (lower panel).

As shown in Fig 2.1, for key parameters, sensitivity between runs with different grid types appears insignificant except for a generally small degradation in wind speed bias with quadratic grid than with linear grid. Similar results are seen in tests for different seasons. In the test on the CARRA-West domain between quadratic and cubic grids, the wind speed error is smaller for quadratic grids. Thus it appears that, going from linear grid to one with a more aggressive spectral truncation, there is a tendency for decreased accuracy in wind speed, whereas the impact on other parameters appears to be minor. From earlier experiences at DMI in testing grid types with higher truncation in dynamics, such a selection has also been beneficial to the stability of the system. As the use of quadratic grid improves delivery speed in the forecast step by ca 10% in comparison to linear grid, and it reduces computational cost, the quadratic grid is selected for CARRA reanalysis.

2.2. Comparison of option OCND2

In the NWP configurations for a subdomain of the CARRA-East domain (Svalbard and northern Scandinavia), a special tuning for parameterization of microphysics for mixed phase (on snow, graupel and rain), "OCND2", has been used, whereas such a tuning has not been used for the CARRA-West domain (Greenland and Iceland) due to concerns on forecasts there for snow and precipitation using earlier HARMONIE-AROME versions. In order to harmonize on this, a test has been configured using 40h1.1.1-based CARRA versions for the CARRA-West domain with OCND2 option turned on and off. As shown in the summary table 2.3 for the key verification parameters (except MSLP), the results with OCND2 is overall more favorable, the latter is hence selected.

Table 2.3: Relative skill scores between runs for CARRA-West domain with and without OCND2 mixed phase microphysics option in CARRA, for 4 test periods. The '+' sign in the table indicate better relative skill, '-' for worse skill, and '=' for similar results.

2.3. Large scale error constraint

For the CARRA reanalysis, hourly ERA5 analyses are used as lateral boundary conditions. In contrast to a limited-area model, which is good in representing meso- and local scales, the ECMWF global model products are known to be superior in representing large-scale information, for example Rossby waves with length scales of 1000 km or more. This is due to the full coverage of all large scales in a global system, a more efficient use of satellite observations and a more advanced data assimilation methodology at ECMWF. In order to take full advantage of large scale information from ECMWF data, not only on the lateral boundaries, the HARMONIE forecasting system has two methods for constraining the initial conditions of the limited area NWP model. The first of these, referred to as LSMIXBC ("Large Scale Mixing with Boundary Condition"), is the default option in HARMONIE-AROME and used in the operational NWP systems. The LSMIXBC scheme provides a weighted average in spectral space of the host model fields and short-range high-resolution HARMONIE forecast fields. The obtained model state is then used as first-guess in variational data assimilation. The second method, referred to as a large scale error constraint, adds an additional term, abbreviated as J_k, to the cost-function of the variational data assimilation. The term measures the distance between the initial conditions and the host model state applying an empirically determined matrix in the norm. During the preparation phase of CARRA, both the LSMIXBC scheme and the large scale error constraint scheme (J_k) were evaluated for use to constrain the initial conditions of the high-resolution regional reanalysis with the information from the global ERA5 data sets. A number of one-month validation experiments for CARRA-East and CARRA-West domains for different periods were set up, see details in Bojarova et al, 2019. From these experiments, it is observed that, the optimally tuned J_k configuration is able to deliver the initial fields for HARMONIE-AROME forecasts of a similar or better quality than the LSMIXBC scheme. Such an improvement is particularly obvious for the upper air humidity fields which are not treated in the current implementation of the LSMIXBC scheme. At the same time a strong impact of the J_k term on the numerical solution of the minimization problem is observed, which suggests needs for a longer quasi-operational monitoring to evaluate and eventually to tune the scheme. However, in view of the project time constraints, it was decided to use the default option for constraint of the large scale, LSMIXBC, in the CARRA production, while the J_k method is to be considered for future versions of reanalysis.

2.4. Selection of climatological structure functions

The background error constraint plays an important role in initializing a NWP model with observations, especially for the Arctic regions considered here where the available number of some observation types is rather sparse. To derive structure functions, the basic procedure in HARMONIE-AROME is to construct an ensemble of data assimilation (EDA) driven by the ERA5 ensembles for a selected period, thereafter to compute differences between ensemble members to sample the time series of forecast differences. The inverse of the square-root of the background error covariance is used to transform model state variables from the "physical space" to the "control vector space", where the elements of the latter are required to be statistically independent. For CARRA, it appears that ca 800 ensemble members are sufficient and adequate to produce a statistically stable estimate of the background error covariance.

In HARMONIE-AROME, two alternative methods to construct EDA for generation of structure function have been tested in order to select an optimal approach. The traditional "EDA" approach inserts uncertainty into observation space, on scales of the observing system, and uses data assimilation as a device to create analysis increments; in this way the uncertainty from observation space is spread into physical space of the model state when propagated through the forecast model runs. The "BRAND" approach, on the other hand, does not perturb observations but samples the uncertainty directly in the control vector space perturbing the whole range of scales. Afterwards the uncertainty from the control vector space is translated into the uncertainty in the physical model space using the background error covariance model in HARMONIE-AROME 3D-Var.

In order to perform a comprehensive evaluation of structure functions derived with "EDA" and "BRAND" approaches, 2 domains x 2 months x 2 derivations = 8 set of the background error statistics were derived. These sets were evaluated both from the physical soundness of the computed structure functions and based on forecast quality of the validation runs (1 month period for winter and 1 month period for summer over both domains) using different sets of background error statistics. From validation against observations comparing the experiments using "EDA" based and "BRAND" based structure functions, the results are mostly of comparable quality, with the experiments with "BRAND" based structure functions generally more favorable for the humidity parameters. An example of the validation against observations for July 2012 over the CARRA-East domain is shown in Figure 2.2. The left plot shows the time, domain and forecast lead time averaged standard deviation error and bias for the forecasts of the relative humidity, while the right plot shows the evolution of the scores with the forecast lead time at 850hPa (the "blue" curves represent the experiments using "BRAND" structure functions, the "green" curve with "EDA"). A reference run experiment, "red" curve, with a preliminary structure function set is plotted for the comparison. As one can see in Fig 2.2b, the relative humidity STD scores for "EDA" experiments show a strong tendency of overfitting at analysis time, followed by a strong degradation with forecast. . On the contrary, the "BRAND" based structure functions allows constraining the analysis field to better fit observations without similar degree of degradation to the forecast

Figure 2.2: Validation against radiosonde data over CARRA-East domain in standard deviation (STD) and bias (BIAS) errors for the relative humidity forecasts from experiments using different structure functions: "BRAND" based structure function ("blue" curve), "EDA" based structure functions ("green" curve), the reference structure functions ("red" curve) from an early first guess set of structure functions. Left plot: the time, domain and forecast lead averaged profile of the scores; Right plot: the evolution of scores at 850hPa along forecast lead time. The validation period is July 2012.

As expected, the background error statistics for the CARRA reanalysis (derived based on both "EDA" and "BRAND" approaches) reveal almost no diurnal dependency, but a strong seasonal dependency, which is associated with different motion scales as well as differences in dominating air masses, and hence weather regimes, during summer and winter seasons.
In order to account for this variation, a smooth transition of climatological structure functions has been constructed for CARRA through a weighting function that changes smoothly over the calendar year. This weighting function is used in defining background error statistics which changes daily by interpolating between the structure functions in winter and summer.

Figure 2.3 shows the validation against radiosonde observations for an autumn month test runs using the smooth transition structure functions ("green" curves) and using winter structure functions ("blue" curves). The reference run structure functions ("red" curves) are plotted for comparison. The results are shown for the CARRA-East domain. Winter structure functions Bwinter are derived from a time series of a "BRAND" ensemble collected over January 2017; Summer structure functions Bsummer are derived from the time series of a "BRAND" ensemble collected over July 2012. The test run is performed over a different period, September 1997. The left plot shows time, domain and forecast lead time averaged STD and BIAS for temperature as a function of pressure levels, while the right plot shows similar scores for the relative humidity. There is a clear positive impact on verification scores, for humidity in particular, from the smooth transition structure functions. These test results indicate that the simple approach with smooth transition of climatological structure functions appears to be able to account for seasonal change in large scale flow characteristics in this reanalysis.

Figure 2.3: Validation against radiosonde data over CARRA-East domain in standard deviation error (STD) and bias (BIAS) for the temperature fields (left) and relative humidity fields (right) from experiments using smooth transition structure functions ("green" curve), winter season structure functions ("blue" curve), the reference structure functions ("red" curve). The time, domain and forecast lead time averaged scores are plotted as a function of pressure levels. The validation period is September 1997

2.5. Glacier snow albedo and snow initialization

The present operational reference version of HARMONIE-AROME does not include a glacier ice model. During summer, melting of snow occurs in the permafrost areas, resulting in large model errors over the glaciers (Mottram et al. 2017). For CARRA the intended climatological applications made it very relevant to improve on this and avoid the unrealistic snow melting. Thus snow depth of glacier grids is initialized at each cold-start and re-initialized every year on September 1st so that the snow water equivalent is set up 9990 kg/m² where the glacier fraction is larger than 0.99. Elsewhere the snow water equivalent is reduced to 100 kg/m² if it is higher than 100 kg/m². Where glacier fraction is present and less than or equal to 0.99, the snow scheme is modified so that the minimum snow water equivalent is a function of the glacier fraction and snow can melt there but never go below the minimum value. On the other hand, no manipulation is done for non-glaciated areas with less than 100 kg/m² snow water equivalent.

To describe as reliable as possible the radiation coupling over the glaciers between surface and atmosphere, the model is adapted to use satellite-derived albedos over the glaciers. These external albedo data are not used directly. Instead, they are used as the old-snow albedo minima in the Douville et al. (1995) snow scheme. Thus, freshly fallen snow in the model can affect the snow albedo and inconsistencies in this regard are avoided. Further details about the revised handling of snow properties over permafrost area and the use of satellite observation derived glacier albedo input are described in the CARRA reanalysis system documentation (Yang et al, 2020 ).

Figure 2.4: Impact of modifications about glacier handling in CARRA on modelled daily T2m in July 2012 as compared to observation data (in blue). The modification (in green) makes snow depth initialization and use MODIS-derived snow albedo data. The CARRA baseline (NWP version) is in red. The data are averaged from analysis time and 3h forecast. Upper left panel is averaged for glacier station network on Greenland PROMICE, upper right for all Greenland surface stations, and lower panel for all Svalbard stations.

Figure 2.4 illustrates the impact of making the described modification on handling of glacier surface in CARRA for a strong melting episode in July 2012. Although the results are still imperfect, the initialization of snow depth and use of satellite derived snow albedo appears to be rather effective in improving the realism of the model.

2.6. High resolution re-processed ocean and sea ice data

CARRA reanalysis uses specially produced high resolution data sets for Sea Surface Temperature (SST) and Sea Ice Concentration (SIC), which are derived from several reprocessed satellite products with both regional (the Baltic Sea) and global coverage. The satellite products are gap-filled Level 4 (L4) interpolated fields, and interpolated onto a 0.05° regular latitude-longitude grid from 56N to 86N and 110W to 90E, covering both model domains in CARRA.

The impact of using high resolution ocean and sea ice data are evaluated against a control configuration for some of the selected test periods. In the control case, SIC and SST data are interpolated from those of ERA5 at coarser resolution. Figure 2.5 shows the comparison for the T2m values along lead time for the month of Sept 1997 for CARRA-East domain, for all stations (Figure 2.5a) and for Svalbard (Figure 2.5b), respectively, indicating an improvement due to use of high resolution datasets with ESA-CCI/OSISAF.

As detailed in Yang et al (2020), the high resolution SST and SIC data in CARRA are combined datasets with the larger part of the reanalysis period based on ESA-CCI data (Toudal et al, 2017), with other datasets filling up gaps. For historical periods without ESA-CCI, OSISAF (Tonboe et al, 2016) data is used as replacement. To check the continuity and time consistency of the resulting CARRA reanalysis during transitions where these input data sets shift, tests have been run to examine whether or not there is any significant sign of changes in surface temperature tendencies, as shown in Figure 2.6 for the month of July 2012, when ESA-CCI data experienced a gap period, during which OSISAF data is used instead. From Figure 2.6, it appears that the main tendencies in the T2m analyses are not sensitive to the minor discontinuity in the CARRA time series of the input SST and SIC.

Figure 2.5: Impact of high resolution ocean and sea ice on analyzed surface temperature in CARRA-East domain for (upper) all stations and (lower) stations in Svalbard, for the month of Sept 1997. The plots are T2m errors in STD and BIAS compared to observations, red for the control and green with high resolution sea states data.

Figure 2.6: Comparison of time series for T2m error in comparison to surface observations in CARRA-West domain, in response to use of OSISAF (in red) and combined ESA CCI-OSISAF data (in green), respectively, for the month of July 2012, which indicates no significant impact due to discontinuity of ocean and sea ice data.

2.7. Impact of surface observation data

One of the significant advantages with the CARRA reanalysis is the use of a large amount of additional surface observation data on top of the GTS data, the latter are the main data used by ERA5. These additional data amounts 3 to 5 times more than those from GTS, enhancing greatly the surface observation data coverage by CARRA for this Arctic region with generally sparse conventional observing network. The collected non-GTS data in CARRA consists mainly of those from national databases of the Nordic meteorological services and other external sources. For the domain of CARRA-West, e.g., additional station data are collected via DMI from the glacier observation networks of PROMICE (Programme for Monitoring of the Greenland Ice Sheet, Denmark), GC-NET (Greenland Climate Network), and the coastal network ASIAQ Greenland Survey.

As an illustration of quality enhancement with CARRA thanks to assimilation of the additional surface observations, in Figure 2.7, results of a sensitivity test with and without use of the non-GTS, additional surface observation data is shown. Here daily averaged time series for STD and BIAS T2m errors with CARRA-West is shown for the 40-day period between 1 Sept and 10 October 2009. In this test non-GTS data is turned off for 20 days between 10 September and 30 September. The remaining periods are all run with full data set with both GTS and additional data sets. The verification in Figure 2.7 is made with different combinations of station lists to illustrate the impact of assimilating additional surface data. Clearly, assimilation of additional observation data has been overall positive for all stations, especially for the glacier areas represented by glacier station list GCNET, PROMICE, which are both poorly modelled and seldom represented by observations in NWP models.

Figure 2.7: Daily averaged CARRA T2m time series between 1 Sept and 10 Oct 2009 for the CARRA-West domain. In contrast to ERA5, CARRA uses large amount of non-GTS surface observation data (in green). In the sensitivity test, these additional data are withdrawn during a 20-day period between Sept 10 and Sept 30 (in red). The plots are from upper left to lower right: a) is averaged for all stations, b) for Greenland stations, c) for Iceland, d) for Svalbard, e) for the glacier network GCNET in Greenland, f) for the glacier network PROMICE.

2.8. Impact of satellite snow information

For CARRA reanalysis, a dataset using the "CryoRisk Snow extent" product using the approach developed at MET Norway (Homleid et al, 2016), which is based on AVHRR data, has been produced. This is available daily at 5 km resolution for the CARRA period (Yang et al, 2020). The data set is activated for the period between March and October each year. In Figure 2.8, impact of the assimilation satellite snow data for CARRA is illustrated by comparing averaged T2m errors in STD and BIAS along forecast lead time for CARRA-West domain during the spring melting season, from 20 March to 30 April 2007. The results show some benefit in T2m temperature for the period, especially for an Iceland station list, indicating benefit of satellite snow information for simulation of melting seasons in Iceland.

Figure 2.8: Averaged CARRA T2m error in STD and BIAS along forecast lead time during 20 March and 30 April 2007 for (upper panel) CARRA-West domain and (lower panel) for an Iceland station list, comparing baseline (in red) with the run using satellite snow observation (in green).

2.9. Impact of Atmospheric Motion Vector data

The impact of atmospheric motion vectors on forecast scores was tested by running assimilation experiments in January 2017. At first, the impact was neutral or negative for wind and temperature when forecasts were verified against radiosondes. After doing Desroziers diagnostics (Desroziers et al, 2005) on the observation minus first guess and observation minus analysis departures it seemed like there was a possibility that neighboring observation errors were correlated. Therefore the thinning distance was increased from 45 to 60 km. The observation errors for AMVs were also modified after comparing the CARRA settings with those of ERA5. After that, the forecast impact became neutral to positive. Experiments were done on both of the CARRA domains and the results were similar. Figure 2.9 shows the verifications for CARRA-East domain.

Figure 2.9: Impact of AMV assimilation on wind speed forecasts for 23 days in January 2017 on CARRA-East domain. Green with AMV assimilated and red for control with conventional observations only.

2.10. Impact of Microwave Sounding Unit (MSU) data

Radiances from the Microwave Sounding Unit (MSU) on the NOAA-11 and NOAA-14 satellites are used from the start of the CARRA dataset in 1997. After spinning up the bias correction coefficients the impact on forecast scores was tested for 30 days in September 1997. The overall impact as shown in Figure 2.10 indicate a neutral impact, with a very slight positive signal on temperature and wind speed compared to radiosondes.

Figure 2.10: Impact of MSU on temperature forecasts for 30 days in September 1997. Green: MSU data assimilated, Red: reference (conventional observations only)

3. Quality assessment of tagged versions of the CARRA reanalysis system

The preparation phase of the CARRA reanalysis system involved a series of tagged versions with the CARRA system repository, taking on board system adaptations and improvements gradually introduced into the reanalysis system during the development work. Table 3.1 lists the tagged versions and main features introduced.

Table 3.1: Tagged system repositories used during preparation and production phases of CARRA. 

For each tagging, extensive validation tests were organized to evaluate the general performance, with necessary corrections and tunings applied where appropriate. This quality assurance is primarily made for the five 1-month-long episodes as previously listed in Table 2.1, representing different seasons throughout the reanalysis period.

With the validation runs with different versions of CARRA, a comprehensive inter-comparison to the in-situ observing has been performed, also with the corresponding global reanalysis ERA5. In this section, some examples of validation results are shown both in form of verification statistics and case studies for high impact events.

3.1. Statistical verification results

Figure 3.1: The daily averaged time series of STD and BIAS errors (Y-axis) by various versions of the CARRA reanalysis in comparison to synoptic stations for the month of (top to bottom) September 1997 (upper panel), January 2000 (mid panel) and April 2007 (lower panel).CARRA versions are shown with different colors in cyan for carra_alpha2, magenta for carra-beta1, blue for carra-beta2, green for carra-rc1, red for CARRA production.

Figure 3.1 shows the averaged daily time series for STD and BIAS in comparison to surface observations for the months of September 1997, January 2000 and April 2007 by various tagged CARRA versions. The results confirm that CARRA reanalysis system indeed progresses gradually in terms of verification scores with each intermediate release, in a rather solid manner. Especially for STD scores, there is in general a tendency of clear improvement from the earliest validated version, CARRA-alpha2, to the final CARRA version. Among these, the improvement in the final version of CARRA is believed to be associated with the activation of non-GTS observation data, which has the clearest impact on screen level temperature.

 
Figure 3.2: The averaged STD and BIAS errors (X-axis) along main pressure levels (Y-axis) for the month of September 1997 by reanalysis in comparison to radiosonde observation for the regions covered by CARRA reanalysis. (Upper left) for temperature in K, (upper right) for wind speed in m/s, (lower left) for geopotential heights in meters and (lower right) for dew point temperature in K. Re-analyses data in comparisons are those by ERA5 (red), CARRA beta1 (green), CARRA_beta2 (blue), CARRA_rc1 (magenta) and CARRA production (cyan).

Figures 3.2 and 3.3 show the validation of the reanalysis datasets against radiosonde data for ERA5 and various versions of the CARRA reanalysis, for the month of September 1996 and April 2007, respectively. While the CARRA fit to observations is generally superior to ERA5, there is also a clear skill improvement between CARRA data from earlier versions towards latter versions. This is particularly evident for the final version of CARRA, which clearly improves on the lower troposphere scores on temperature, dew point and wind speed. The improvement is believed to be mainly due to the updated structure functions used in 3D-Var .

Figure 3.3: Same as Figure 3.2 but for the month of April 2007.

In the following, we examine the performance of CARRA in some high impact weather situations, also in comparison to the corresponding ERA5 host model data set.

3.2. Simulation of the Icelandic storm on March 14 2015

A low pressure storm system passed through Iceland rapidly in less than half a day on March 14, 2015. Figure 3.4 shows the simulated maximum wind by CARRA (a) and ERA5 (b) at 9 UTC respectively. At that point in time, the low is just west of Iceland with a very strong wind field coming from the south west. One notices the much higher wind speeds in CARRA than in ERA5, and in particular in the interior of Iceland. Maximum wind speed in CARRA is 46 m/s in the inland, while ERA5 maximum is 26 m/s at the south west coast. Compared to surface station observations CARRA has a positive bias (+2.3 m/s), while ERA5 heavily underestimates the maximum (-8.6 m/s). The correlation between the Icelandic observations and reanalysis over the storm period shows that CARRA (0.61) is in better agreement with the observations than ERA5 (0.23). In summary, CARRA captures the strength and spatial variability in W10m better than ERA5 for this high impact storm.

Figure 3.4: MSLP and WS10m from CARRA (left) and ERA5 (right) analyses on 14. March 2015 09 UTC. Local maxima in W10m and MSLP minima are given in blue and black numbers, respectively.

3.3. Simulation of a landfall of the polar low at the Norwegian coast

It has been reported that the wind speed in ERA-Interim (we are not aware of similar studies of ERA5) are heavily underestimated (Spengler et al., 2017). Based on the polar low climatology of Noer et al. (2011) it is estimated that there is a meso-scale disturbance developing into an intense polar low south of Svalbard in the morning of 6. April 2007. At 06 UTC the system had a central pressure and maximum wind speed of 975 hPa and 27 m/s in CARRA and 980 hPa and 24 m/s in ERA5, both in approximately the same position (~72.2N, 11E). The polar low made landfall 12 h later, close to Lofoten in Northern Norway (Figure 3.5). In Bodø, the second largest city in northern Norway, maximum measured wind speed was 21,7 m/s with gusts on 28,4 m/s. Comparing maximum wind speed from observation stations in the vicinity of the landfall reveals that CARRA is in better agreement with observations for this event (bias 0.0 m/s, STD 3.5 m/s, correlation 0.73), than ERA5 (bias -2.6 m/s, STD 4.3 m/s, correlation 0.57). Going through the polar low events as reviewed by Noer et al. (2011) for the winter of 2006/2007, CARRA is found to show a generally higher wind speed than those with ERA5. In summary, CARRA simulates a slightly deeper polar low with higher wind speeds than ERA5, and hence in better agreement with observed wind than ERA5.

Figure 3.5: MSLP and 10m wind speed from CARRA (left) and ERA5 (right) for 6. April 2007 18 UTC.

3.4. Simulation of a winter rain event at Svalbard

A strong transport of heat and moisture northward created a winter rain event (temperatures well above 0°C at the lower elevations) at Svalbard in the beginning of January 2016. The situation lasted for several days before temperatures rapidly dropped down to ~ -10°C. This type of events have a substantial impact on infrastructure, society, and wildlife (Serreze et al. 2015; Hansen et al. 2014). A distinct difference in the spatial distribution of the precipitation is seen between CARRA and ERA5 due to their very different resolution (Figure 3.6). At Ny-Ålesund, 26,7mm is measured in 24 h, in comparison to a normal precipitation in January for 32.0mm. The corresponding simulation for this event in the reanalyses is 15.4mm and 12.0mm in CARRA and ERA5, respectively. At Svalbard Airport, 9.2 mm is observed compared to a normal precipitation in January at 15.0mm, and CARRA simulation varies between 6.4 and 13.4 mm between the four closest grid points, while ERA5 forecasted 11.5 mm. Even though it is not evident that CARRA is in substantially better agreement with observations, it appears to have a more realistic spatial variability, as shown by Figure 3.6.

Figure 3.6: Accumulated precipitation (mm/24 h) from 03. January 06 UTC to 04. January 06 UTC from CARRA (left) and ERA5 (right).

4. Concluding remarks

Through examination of model simulations in the high impact weather cases as shown in the above sections, it is clear that the high resolution CARRA reanalysis shows clearly added value in simulation of high impact weather events which can be critical in Arctic climate. No doubt, these advantages shown by the CARRA reanalysis in high impact cases contribute to the overall verification inter-comparison against ERA5, as shown in the CARRA system documentation (Yang et al, 2020).

In summary the high resolution CARRA reanalysis appears perform better than ERA5 in representation of high impact weather and in general and provide improved climatology for the CARRA domains.

5. References

Bengtsson, L., U. Andrae, T. Aspelien, Y. Batrak, J. Calvo, W. de Rooy, E. Gleeson, B. Hansen-Sass, M. Homleid, M. Hortal, K. Ivarsson, G. Lenderink, S. Niemelä, K.P. Nielsen, J. Onvlee, L. Rontu, P. Samuelsson, D.S. Muñoz, A. Subias, S. Tijm, V. Toll, X. Yang, and M.Ø. Køltzow, 2017: The HARMONIE–AROME Model Configuration in the ALADIN–HIRLAM NWP System. Mon. Wea. Rev., 145, 1919–1935, doi:10.1175/MWR-D-16-0417.1

Bojarova, J., X. Yang, M. Dahlbom, H. Schyberg, O. Vignes, 2019: Analysis of approaches for large scale information inclusion and decision for method. Copernicus Climate Change Service contract 2018/C3S_D322_Lot2_METNorway/SC1 deliverable report no C3S_D322_Lot2.1.1.1.

Desroziers, G., Berre, L., Chapnik, B. and Poli, P., 2005: Diagnosis of observation, background and analysis‐error statistics in observation space. Q.J.R. Meteorol. Soc., 131: 3385-3396. doi:10.1256/qj.05.108 .

Douville, H., Royer, J.-F., and Mahfouf, J.-F., 1995: A new snow parameterization for the French community climate model. Part I: Validation in stand-alone experiments, Clim. Dynam., doi:10.1007/BF00208760, 12, 21–35.

Hansen, B. B., and Coauthors, 2014: Warmer and wetter winters: Characteristics and implications of an extreme weather event in the High Arctic. Environ. Res. Lett., 9, 114021, https://doi.org/10.1088/1748-9326/9/11/114021.

Homleid, Mariken and Mari Anne Killie, 2013: HARMONIE snow analysis experiments with additional observations, MET report 06/2013, available on https://www.met.no/publikasjoner/met-report/met-report-2013.

Høyer, J. L., 2016: Product user Manual for Baltic Sea SST Level 4 Product SST_BAL_SST_L4_NRT_OBSERVATIONS_010_007_b, CMEMS user manual: [online] Available from: http://cmems-resources.cls.fr/documents/PUM/CMEMS-OSI-PUM-010-007-b.pdf (Accessed 10 September 2017).

Høyer, J. L. and Karagali, I., 2016: Sea Surface Temperature Climate Data Record for the North Sea and Baltic Sea, J. Clim., 29(7), 2529–2541, doi:10.1175/JCLI-D-15-0663.1.

Køltzow, M., B. Casati, E. Bazile, T. Haiden, and T. Valkonen, 2019: An NWP Model Intercomparison of Surface Weather Parameters in the European Arctic during the Year of Polar Prediction Special Observing Period Northern Hemisphere 1. Wea. Forecasting, 34, 959–983, https://doi.org/10.1175/WAF-D-19-0003.1

Mottram, R., Nielsen, K. P., Gleeson, E., and Yang, X., 2017: Modelling Glaciers in the HARMONIE-AROME NWP model. Adv. Sci. Res., doi:10.5194/asr-14-323-2017, 14, 323–334.

Nielsen, K. P., Yang, X., Agersten, S., Støylen, E. and Schyberg, H., 2019: C3S Arctic regional reanalysis - Data User Guide. Copernicus Climate Change Service contract 2018/C3S_D322_Lot2_METNorway/SC1 deliverable report no C3S_D322_Lot2.1.1.1. Available at Copernicus Arctic Regional Reanalysis (CARRA): Data User Guide

Serreze, M. C., A. D. Crawford, and A. P. Barrett, 2015: Extreme daily precipitation events at Spitsbergen, an Arctic Island. Int. J. Climatol., 35, 4574–4588, https://doi.org/10.1002/joc.4308.

Spengler, T., C. Claud, and G. Heinemann, 2017: Polar Low Workshop summary. Bull. Amer. Meteor. Soc., 98, ES139–ES142, https://doi.org/10.1175/BAMS-D-16-0207.1.

Tonboe, R. T., Eastwood, S., Lavergne, T., Sørensen, A. M., Rathmann, N., Dybkjær, G., Pedersen, L. T., Høyer, J. L. and Kern, S., 2016: The EUMETSAT sea ice concentration climate data record, The Cryosphere, 10(5), 2275–2290, doi:10.5194/tc-10-2275-2016.

Toudal Pedersen, L., Dybkjær, G., Eastwood, S., Heygster, G., Ivanova, N., Kern, S., Lavergne, T., Saldo, R., Sandven, S., Sørensen, A. and Tonboe, R., 2017: ESA Sea Ice Climate Change Initiative (Sea_Ice_cci): Sea Ice Concentration climate data record from the AMSR-E and AMSR-2 instruments at 25km grid spacing, version 2.0, , doi:10.5285/c61bfe88-873b-44d8-9b0e-6a0ee884ad95.

Yang, X., Nielsen K.P., Dahlbom, M., Amstrup, B., Peralta, C., Høyer, J., Englyst, P.N., Schyberg, H., Homleid, M., Køltzow, M.A.Ø., Randriamampianina, R., Dahlgren, P, Støylen, E., Valkonen, T., Palmason, B., Thorsteinsson, S., Bojarova, J., Körnich, H., Lindskog, M., Box, J., Mankoff, K., 2020: C3S Arctic regional reanalysis – Full system documentation. Copernicus Climate Change Service contract 2018/C3S_D322_Lot2_METNorway/SC1 deliverable report no C3S_D322_Lot2.1.1.1. Available at Copernicus Arctic Regional Reanalysis (CARRA): Full system documentation

6. Related articles

• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): Data User Guide
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): known issues and uncertainty information
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): Uncertainty estimation method
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): Complete test and verification report on fully configured reanalysis and monitoring system
• Page:
  Copernicus Arctic Regional Reanalysis (CARRA): Full system documentation