Contributors: UNIVIE: Leopold Haimberger, Federico Ambrogi, Michael Blaschek, Ulrich Voggenberger
Issued by: UNIVIE / Leopold Haimberger
Issued Date: 30/06/2021
Ref: C3S_ DC3S311c_Lot2.2.1.4_ATBD_Temperature_Humidity_Adjustments
Official refence number service contract: 2019/C3S_311c_Lot2_UNIVIE/SC1
1. Introduction
This document describes the homogeneity adjustment software for developing the adjustments provided by this service. They are based on published methods, which have been updated to use more modern reference data.
For temperature adjustments we refer to Haimberger (2007), Haimberger et al. (2012). The temperature adjustments are based on RAOBCORE/RICH, still for 00GMT and 12GMT. The method has been enhanced by taking into account the annual variation of the solar elevation and through the use of better input- and reference data (ERA5 (Hersbach et al. 2020, Bell et al. 2021) background departures after 1950, departures from NOAA 20CRv3 reanalysis before that).
Humidity adjustments were calculated using the method first described in D4.2 of EU-project ERA-CLIM2 and also in MC3S311c_Lot2.2.1.2. Similarly to the RAOBCORE temperature adjustment, it uses background departure time series as means for detecting and adjusting biases. Dai et al. (2011) reported on biases clearly detectable in DPD and showed that this variable is well suited for break detection. Tests on adjustment performance indicate that using either DPD or RH give reliable results. The adjustment algorithm is based on quantile matching, since in contrast to temperature measurements, the shape of the measurement distribution can change strongly between different sections of the time series. It is assumed that the measurement distribution or the background departure distribution of the most recent part of the time series is most realistic, and the other sections of the time series are adjusted such that they match those distributions. If the most recent parts of the time series consists of measurements from high quality radiosondes such as Vaisala RS92 or RS41 based on metadata information, no adjustments are performed. Humidity bias adjustments are described in sections 6-8.
It has been a design goal that CDS users can reproduce the adjustments downloaded from the CDS based on the observations and background departures, also delivered by the CDS. Open-access to routines and reproducibility of results ensure quality of the service. The source codes are all public and the processing chain starts with downloading the input data from the CDS frontend. The software then calculates the adjustment and writes them back into files with a legacy format, as it has been used for the ERA5 assimilation input, but also into netCDF files which are accessed by the CDS backend. Reproduction is most straightforward for wind data. Only some adaptation of paths to the user's need are likely necessary. It is more complex for humidity and even more so for temperature, since temperature homogenization requires the download of all station data and it is written in FORTRAN. Nevertheless, it is in principle feasible. The only obstacle is that not all the observation data that are used for calculating the adjustment could be made public on the CDS for data licensing reasons. The users can thus calculate adjustments that usually come close to but are not the same as the adjustments provided via CDS. The results of this reproduction exercise can be found in DC3S311c_Lot2.3.2.2.
The document first shortly describes the merging procedure for the observation and background departure records and then describes first the temperature adjustments and their impact on the data, and afterwards the humidity adjustments and their impact on the global radiosonde network.
2. Merging procedure
The merging procedure is described in detail elsewhere (DC3S311c_Lot2.1.4.2 and Product User Guide). We just note here that the merged data base contains data from ERA5 observation data base as main data source, with additions from IGRA2 and the NCAR UADB and the early upper air data collected in ERA-CLIM2. It also contains digitized data from radiosonde intercomparison campaigns (Imfeld et al. 2021).
The merged data base is the most comprehensive upper air data collection for assimilation. Unfortunately, not all data could be guaranteed to be freely distributable to users, particularly data from the ERA5 observation data base. It has been decided to make available only those records of the observation data base which are available also in other public archives such as IGRA2 and NCAR UADB. For more than 93% of records such matches could be found. In DC3S311c_Lot2.1.4.2 it is shown that data particularly from the 1960s cannot be made public, but there are also a few records in more recent time. The following plots except Fig. 1 are based only on those data which are also freely available.
|
|
Figure 1: Time series of monthly radiosonde temperature record numbers available at the 500 hPa level, south of 20N (left), north of 20N (right). Blue line denotes number of homogenized records - 2012, the time of the last major update of the RAOBCORE data set. Orange line is the number of radiosondes available - 2020, which are also homogenized. Numbers in the current service (RISE) are lower in the 1970s and 1980s due to some duplicates removed.
The data base contains data on standard pressure levels, but also on height levels (PILOT stations, not relevant here) and on significant pressure levels, at synoptic times (12GMT and 00GMT) but also asynoptic times. Time series of the number of stations with data for a particular month as they were available for Haimberger et al. (2012) and as they are available in the present data set are shown in Fig. 1. Long records suitable for climate analysis can only be generated from standard pressure level data at fixed launch times. Therefore the homogeneity adjustment procedure was performed only with standard pressure level data in a time window of +/-3 hours around the 00GMT and 12GMT times. That may still appear generous, but it ensures that long time series exist even if there were changes in launch schedules. Since the data are assimilated at their appropriate time, a shift in time schedule alone does not cause a break. Only if the bias changes due to the different launch schedule and thus different solar elevation angle, there will be a break. Changing launch times may have an effect on calculated trends, however, at pressure levels with a strong diurnal cycle, particularly at 850 hPa.
Maps of trends in Fig. 2 give some indication on both data amount and trend heterogeneity. The temperature trends are from unadjusted radiosondes for 1954-1974 at 300 hPa, for 1964-1984 at 100 hPa (comparison with Haimberger et al. 2012) and 1979-2006 at 100 hPa and 1979-2019 at 100 hPa. Note spurious cooling trends over the Former Soviet Union in the period 1954-1974 and over Central Europe, China and Australia in the later periods.
|
|
|
|
Figure 2: Linear Temperature Trends from unadjusted radiosonde time series at different intervals and pressure levels. Bullets are plotted only if at least one of the 00GMT and 12GMT time series has less than 24 months missing in the considered intervals.
3. Building appropriate reference series for homogenization
The RAOBCORE/RICH approach detects breaks in the observation from time series of departures to suitable reference series. The choice of reference series is by far not obvious. Temperature time series from full reanalysis background forecasts are a good choice for the satellite era and have been used successfully (Haimberger et al. 2008, Haimberger et al. 2012). While for the EU project ERA-CLIM a complicated concatenated background had to be chosen (involving ERA-Interim, JRA55 and an experimental assimilation going back to 1939 (Hersbach et al 2017), we could use ERA5, including its (preliminary) backward extension to 1950 (Bell et al. 2021) as background. For observations reaching back even further, we decided to use the NOAA 20th century analysis version 3 (NOAA 20CRv3, Sivlinski et al. 2019) as reference.
In addition, analysis feedback data must be added, particularly for those data which have not been assimilated in ERA5. For the ERA5 period, the background departures are calculated offline on standard pressure levels from the available gridded ERA5 background forecast data. For the pre-1950 period the NOAA 20CR v3 (Slivinski et al. 2019) is used as background. The interpolation error is larger than during online assimilation because of larger time and space deviations (1 deg grid, 6-hourly analyses/forecasts), but particularly in the early days, this error appears not to be larger than the forecast errors. In order to avoid jumps in the mean of the background time series, we added the difference between ERA5 and the NOAA 20v3 for the period 1950-1951 to the NOAA 20v3 temperatures from before 1950. This strongly reduces the jump that can be detected in the year 1950.
Figure 3: Temperature departure time series for station Bethel (Alaska, 70219) at the 100 hPa level at 00GMT (green) and 12GMT (blue). Departures are taken from ERA5 (from 1950 onward) or NOAA 20CRv3 (up to 1950). A 30 day running mean is applied, the figures in the legend are standard deviations of the time series with the 30 day running mean applied.
Figure 3 shows background departure time series for station Bethel at 00GMT and 12GMT. The daytime series (00GMT for a station in Alaska) exhibits big changes in the background departures, which can be traced back to instrument changes at that time. The time series goes back to before 1950, where the background is from the NOAA 20CR v3 reanalysis.
Overall, the departures have higher variance before 1950, which can be seen from the departure time series in Figure 4. This is expected, since there are only few stations available and since a surface data only reanalysis is less accurate than a full reanalysis that assimilates upper air data.
Figure 4: Average of departures from all available radiosonde stations at 00GMT and 12GMT at 200 hPa, where stations had both 00GMT and 12GMT ascents. Departures are from NOAA 20CRv3 (before 1950) and ERA5 (1950 onward). Red curves are time series of SNHT test statistics (Haimberger, 2007). Number of records is ~500 in the most recent part, but before 1950 this number goes down to less than 100 (see Fig. 1).
4. Temperature homogeneity adjustment procedure
Homogenization is performed in four steps:
- Breakpoint analysis is performed on observation minus background difference series as described in Haimberger (2007), using the Standard Normal Homogeneity Test modified such that seasons before/after a potential breakpoint are equally sampled. The difference series as shown in Figure 3 are examples. In addition, Metadata obtained from Schroeder (2008) are considered by raising the a priori probability for breakpoints in case of documented instrument or radiation correction changes (see Haimberger (2007) for details. The metadata can be retrieved via the CDS (see Product User Guide).
- Homogeneity adjustments are calculated from
- Difference series used also for break detection (RAOBCORE method)
- Difference series between observation records and a composite record calculated from pieces of neighbouring radiosonde time series (RICH-obs, see Haimberger et al. 2012)
- Difference series between obs-bg difference series at tested site and difference series at neighbouring stations (double differencing approach, RICH-t, see Haimberger et al. 2012)
- Observation time series as a whole are shifted such that the mean difference of the most recent part of the time series to a reanalysis reference corresponds to a composite of differences to this reference calculated from neighbouring time series. The time interval used is 8 years unless the series to be shifted is shorter. Time series are shifted if
- they end before 2009
- the most recent years of the record consist of measurements from radiosonde types suspected to be biased. Vaisala RS90, RS92, RS41 and most recent Japanese Meisei radiosondes are considered unbiased and are therefore not shifted.
- The adjustments just described are all constant in time between breakpoints. However, the radiosonde observation bias often has a seasonal cycle related to the seasonal change in solar elevation. This effect is particularly noticeable at high latitudes and at longitudes around 100W and 80E, i.e. when 00GMT or 12GMT occur at dawn or dusk. The annual cycle of departures between the reference (ERA-Interim or JRA55) and the adjusted radiosonde time series is estimated by calculating monthly means of the departures. The annual cycles of each interval between breakpoints are analysed since radiosonde observations likely typically have different annual cycles of the bias before and after a breakpoint. Then a possible mean of the departures is subtracted so that the removal of the annual cycle does not change the mean temperature of the interval considered, which is important, because RAOBCORE/RICH have already adjusted the mean. This way a "double adjustment" is avoided. Figure 7 gives an example how the adjustments look like. This solar elevation dependent adjustment is applied only if the departures are strongly positively correlated with annual variations of solar elevation angle (threshold 0.7, must occur at least at 3 levels). It helps to reduce the rms of the departures from reanalyses.
In order to run the executable with the available observations, one first has to convert the data retrieved from the CDS with a simple script adjust/Converters/from_cds_to_legacy.py.
The RAOBCORE/RICH software reads those files together with a few auxiliary files adjust/RISE_FORTRAN/tables and calculates the adjustments. The RAOBCORE/RICH software requires that all stations are available in the legacy format. Also, it requires substantial resources (30 GB memory, 30 CPU core hours on modern hardware). RAOBCORE/RICH yields adjustments (RAOBCORE_bias_estimate and RICH_bias_estimate) that are constant between breaks. The codes for RAOBCORE/RICH can be found at adjust/RISE_FORTRAN. It runs on Linux platforms and can be compiled with either the Intel (recommended) or gfortran compilers. In addition, netCDF libraries with a Fortran interface must be installed.
With the script adjust/Converters/add_solarangle_adjustments.py the solar-elevation dependent adjustments (RASE_bias_estimate and RISE_bias_estimate) are then created. The script produces the adjustment files in the same format as it was used as input for assimilation with ERA5. These files, which are CF compliant but not CDM compliant, can be downloaded via https://srvx1.img.univie.ac.at/webdata/haimberger/RAOBCOREv1.7/
Finally, in order to write back the adjustments also to the service's central database of merged files into the 'advanced_homogenisation' table, the script adjust/Converters/adjustments_from_legacy_to_cds.py is run. This script is given here only for documentation. It can be run only at places where the files of this data base are directly accessible, e.g. on the virtual machine (VM) where the backend for this service resides. Once the merged files are uploaded to the CDS VM, they can be accessed via the CDS using one of the optional keywords. So far, the adjustments are only calculated for standard pressure levels. The users may interpolate them in the vertical to temperatures at other than standard levels. This may be changed later if there is high user demand for this.
Figure 5: Adjustments from RASE and RISE for station Bethel (Alaska) at 100 hPa level at 00GMT and 12GMT. The variation of the annual cycle of the bias is only taken into account after 1979, and only if it is clearly detectable from the departure time series between two breakpoints.
Figure 6: Background departures at station Bethel (Alaska) at the 100 hPa level after subtracting RISE bias estimates (shown in Fig. 5). Note substantial decrease of departure standard deviation compared to Figure 3.
The break detection efficiency depends, however, not only on the reference series used but also on the thresholds used for break detection. Figure 5 compares the number of breaks found in the global radiosonde network in each month. More conservative (higher) values would lead to much less detected breaks. The number of detected breaks is, however, not the most important property. Using a homogeneous reference series for adjustments is most essential.
Figure 7: Number of breaks per month for the whole radiosonde network, detected with RAOBCORE v1.5 (Control) and CUON.
5. Effect of temperature adjustments on the whole network
While a more complete scientific documentation of the quality of the bias adjustments is planned in peer reviewed journal publications, the plots below show that for all the variables discussed so far, the spatial heterogeneity of trends, measured as a ''trend cost'' function (see Haimberger, 2007) decreases when applying the homogeneity adjustments. An ipython notebook used to produce those plots using the CDS frontend can be found at adjust/Notebooks/Trend_heterogeneity.ipynb. It works directly with the subdaily temperature data that can be retrieved from the CDS. Figure 8 shows plots obtained with this notebook for the period 1979-2019 for unadjusted data as well as for data with adjustments from this service applied.
The temperature records and adjustments have also been backported to legacy format, in order to compare them with previous RAOBCORE/RICH versions Haimberger et al. (2008, 2012). This also allows to produce gridded monthly means, which have so far not been part of the Copernicus Upper Air Network Service. In particular, one wants to see the effect of the adjustments on tropical surface temperature trend amplification. This was investigated in the above papers but also by Steiner et al. (2020), using RAOBCORE/RICH versions 1.5.1, and 1.7.
Figure 9 shows the same trend homogeneity maps as Figure 2, but with RASE adjustments applied. Figure 10 displays this information for the earlier periods adjusted with RISE. Note the much weaker cooling trends over Europe, Australia and China in the satellite periods shown, and the much weaker cooling trends over the Former Soviet Union in the periods 1954-1974 and 1964-1984. The trend heterogeneity is reduced substantially by the adjustments at all intervals shown.
|
|
|
|
Figure 8: Temperature trends in units K/10a (K/10 years) 1979-2019 at 100hPa from a) unadjusted observations, b) observations adjusted with the adjustments stored in ERA5 analysis feedback, c) RASE v1.8 adjusted observations and d) RISE v1.8 adjustments. These are the adjustments used by CUON. Only time series where values in the first and last 2 years of the interval were available, and where at least half of the observation days were available have been included. Note trend heterogeneity cost function values. The smaller, the better.
|
|
|
|
Figure 9: Temperature trends of time series adjusted with RASE in units K/10a (K/10 years) for different time intervals, using the legacy plot routines. Time series were first monthly averaged and 00,12GMT treated separately. Plots are shown here to permit comparison with earlier results in the literature
|
|
Figure 10: Same as above, for the early period, adjusted with RISE.
|
|
Figure 11: Unadjusted (blue) and adjusted global and tropical belt mean temperature anomaly at 100 hPa. Data have been gridded to 10x10 degree gridboxes, then zonally averaged, then belt averaged. Note that in the early part there are really only 2-3 stations representing the whole tropics.
After gridding, we can calculate also global belt means of adjusted and unadjusted temperatures. These are shown in Figure 11 for the 100 hPa level. One can see the strongest effect of the adjustments in the 1960s and 1980s. Many climate scientists are interested in the vertical profile of belt mean temperature trends, as depicted in Figure 12 and Figure 13. The period 1979-2006 was chosen since it saw the transition from radiosonde types with relatively strong radiation errors to more modern systems such as Vaisala RS80 and was thus affected by large spurious trends. Also this period was considered in earlier publications about RAOBCORE/RICH. The adjustments lead to enhanced warming in the upper troposphere and reduced cooling in the stratosphere, most clearly seen for the Tropics. This has not changed much since the publication of RAOBCORE/RICH v1.4 (Haimberger et al. 2008). One can see, however, that there is less discrepancy between RASE/RISE (RAOBCORE/RICH) adjustments from the CDS compared to RAOBCORE v1.4. Also the trend profiles follow more the shape of a moist adiabatic lapse rate trend profile, as commonly expected (Santer et al. 2005, Steiner et al. 2020). This is seen as a clear improvement compared to RAOBCORE v1.4.
|
|
|
|
Figure 12: Global and tropical belt mean trends for RAOBCORE v1.4 (Haimberger et al. 2008, upper panels) and CUON (lower panels), for period 1979-2006. Black dot is tropical mean surface temperature trend from the HadCRUT4 surface temperature data set, sampled at 10x10 grid boxes where radiosonde data were available.
|
|
Figure 13: Tropical belt trends for interval 1958-2010, for RAOBCORE v1.4 (left) and CUON (right).
Of course it would now be interesting to compare those results with independent satellite data or other reanalyses and observation data. This is however beyond the scope of an Algorithm Theoretical Basis Document.
6. Radiosonde humidity adjustment method
Haimberger (2007) and Haimberger et al. (2012) and the above text have shown how departures between forecasts from the assimilating model of a reanalysis effort and the observations can be used to detect and adjust radiosonde temperature inconsistencies. The described method adjusts the mean differences between samples. For humidity data, it is necessary to extend the method in order to adjust also the shape of the data distributions.
6.1. Data source and internal consistency checks
The data sources are the same as for radiosonde temperature. The primary observation variables were dewpoint temperature and relative humidity. Whenever one humidity variable was available, it could be converted to dewpoint temperature, dewpoint depression, relative humidity and specific humidity. This way one obtains equally long time series for each humidity variable. It can be detected from the CDM-OBS variable 'observations_table/conversion_flag' if a specific value was already available in the data source (0) or has been converted from dew point depression(2), relative humidity (3) or specific humidity (4). The conversion formulae were taken from the IFS reference manual, to ensure maximum consistency with the humidity values available in the ERA5 analysis feedback. The primary variable for break detection and adjustments was relative humidity. In addition, background departures have been calculated at least for relative and specific humidity.
Some humidity measurements from the Global Reference Upper Air Network (GRUAN Dirksen et al., 2014) are of high quality, with comprehensive documentation, and have a focus on climate applications. They are available from 2006 onward and can be downloaded from https://cds.climate.copernicus.eu/cdsapp#!/dataset/insitu-observations-gruan-reference-network. Humidity data from the operational radiosonde network and particularly earlier humidity measurements require homogenization.
6.2. Breakpoint detection & adjustment
An example is station Norman (OK, 72357), whose dewpoint depression and relative humidity time series, together with background departures from ERA5 are shown in Figure 14. Breakpoints are detected from daily-observed DPD minus ERA5 background DPD time series at 00GMT and 12GMT. We apply a Standard Normal Homogeneity Test (SNHT) with a threshold of 50, similar to (Haimberger, 2007). The detection is done for each pressure level individually and only if a significant breakpoint is found at more than 3 levels it is considered a 'realistic' breakpoint, following the hypothesis that an instrumental bias or change would result in a shift at multiple levels. The detection algorithm finds maxima in the summed up SNHT and evaluates how many individual levels have significant peaks, thereby selecting the exact timing of the breakpoint. The specific threshold is set to 50, which is higher than required if the difference time series is uncorrelated and normal distributed between breakpoints to account for autocorrelation and non-Gaussianity. The described method assumes that breaks are in the observations time series, breaks in the ERA5 reference time series cannot be excluded. So far, we have found little evidence of jump-like inhomogeneities in ERA5 humidity variables. Inhomogeneities may be induced by changes in the satellite observing system. In theory, every humidity variable (e.g. RH, VP, dew point or DPD) qualifies in its absolute or relative measure, e.g. departure from ERA5, for the adjustment procedure. In an earlier version of the adjustment procedure, developed in ERA-CLIM2 (Haimberger and Blaschek, 2018), DPD was used. We found, however, that relative humidity can be used equally well, and used relative humidity as variable to be adjusted in this service. For the adjustment of the breaks at each breakpoint, one can use different approaches. One is to calculate mean differences between samples before/after a breakpoint, as it is done for radiosonde temperature homogenization (Haimberger, 2007). For humidity adjustments, however, it is more suitable to calculate differences between quantiles of the samples before/after a breakpoint.
The samples before the breakpoint are then adjusted such that the quantiles of relative humidity (we use up to 50) match those of the sample after the breakpoint. To achieve this, the values in each quantile range are adjusted by the difference between the corresponding quantiles. This procedure is often referred to as quantile matching and allows adjustments to the mean as well as extremes, yielding usually lower adjustments for high relative humidity values and larger adjustments for low relative humidity values. Of course the adjustments must not push the adjusted relative humidities beyond the \[0,1\] interval. The comparison of observation samples from different intervals can be problematic, particularly if the intervals are short and if the differences in the distributions are large. Therefore a third approach is to match the distributions in an interval to the humidity distribution in ERA5, using the quantile matching procedure. However, direct usage of ERA5 humidity is not ideal since itself has biases (e.g. moist bias above the boundary layer etc.). Thus we first match the quantiles of the most recent (after the last breakpoint) section of the observed values with the distribution of humidity in ERA5 for the full time series, creating a quantile adjusted time series of ERA5 so called ERA5-adjusted. Then we can apply quantile matching to observed humidity distributions of earlier periods such that the resulting distribution comes close to the ERA5-adjusted distribution of the most recent period. This method yields the best results, but it is clear that all discussed adjustment methods rely heavily on the long-term stability of ERA5.
The observation time series are processed going back in time and split the time series into segments. From these segments the samples for calculating adjustments are drawn. The adjustment time series for relative humidity at Norman is shown in Figure 14.
Figure 14: Time series of dewpoint departures (upper panel) and relative humidity (lower panel). Shown are observation (blue), background departure from ERA5 (green) and bias estimates (ochre), all averaged with a 30 day running mean, for station Norman (72357) at 300 hPa level. Data do not seem reliable at this level before 1965, thus have been omitted in lower panel. Bias estimates only calculated for relative humidity. Outliers with background departures above/below the upper/lower quartile +/- 1.5 the IQR have been eliminated._
7. Humidity adjustment results
The homogenization algorithm outlined above is fully automatic and does not rely on metadata. However, its performance depends on several parameters (e.g. thresholds for break detection, choice of minimum length of reference series, number of quantiles etc.). We have done some initial tests with different parameters and detection efficiency does not change much, however given time and resources at hand it was not feasible to investigate the whole parameter spectrum and estimate uncertainties in derived quantities. Here we follow expert knowledge and proven values for certain parameters as found in Haimberger (2007). The following results are from one possible realization of the adjustment algorithm. At a later stage, the structural and parameter uncertainties of the adjustment method should be assessed using an ensemble of adjustments. This may be undertaken in a future version of this data set.
The effect on the distribution of relative humidity measurements at different time intervals can be seen exemplary in Figure 15. The relative humidity distribution in the most recent period is skewed towards very low values, at least at the considered station and pressure level, but taken with a modern instrument, so that it is trusted. The distributions from raw observations in the earlier periods, taken with other humidity sensors, look quite different. Subtracting the calculating adjustments from the humidity observations leads to distributions which are much more similar to the most recent distribution.
Figure 15: Histograms of relative humidity at station Norman (OK, 72357) at 300 hPa for different time intervals. Blue=unadjusted, brown=adjusted, semi-transparent colours are used. Number of valid observations differ in intervals.
7.1. Effect on Trends in the station network
One major application of upper air measurements is to calculate global trends. The applied adjustment methods manage to reduce these unrealistically strong RH trends in the troposphere in the interval shown in Figure 16. The unadjusted trends at for example 300 hPa exhibit strong regional discrepancies, showing large anomalies across North America, China and some parts of Europe. The applied adjustments exhibit spatially more consistent trends. The unrealistically strong drying trends in North America, China and Europe are much weaker after the adjustment. While we are sure that the homogeneity adjustments applied are beneficial, they have yet to be rigorously compared to adjustments applied in earlier studies, such as Dai et al. (2011) and Madonna et al. (2020a,b). One should be cautious using those records for climate change analysis, even after adjustment, since particularly before the 1990s, there are serious issues with instrument sensitivity at higher levels. For example, the histogram bar near 0.0 in the rightmost panel of Figure 14 stems from dewpoint depression values deliberately set to -30K under cold dry conditions at this station at this time (see also Dai et al. 2011). Expert knowledge is still required to decide which instruments to consider for climate change investigations.
Figure 16: Relative humidity trends in units %/10a at 300 hPa (upper panels) and 500 hPa (lower panels) from unadjusted observations (left panels), b) observations with relative humidity bias estimate subtracted (right panels). Observation count criteria have been relaxed (30 obs/year for all except 2 years needed for a station to be shown as bullet in the above plots) compared to temperature trend plots above (180/year), since particularly at 300 hPa, many humidity observations are missing.
8. Outlook
The method has yet to be published in a peer reviewed journal, and comparisons with similar data sets need to be performed.
9. Acknowledgements
This work has been funded by Copernicus contract C3S 311c Lot2, some of the methods used have already been developed in EU 7FP projects ERA-CLIM and ERA-CLIM2. The HadCRUT v4 dataset is produced by the University of East Anglia in collaboration with the MetOffice Hadley Centre.
10. References
Andersson, E., Hólm, E. V., Bauer, P., Beljaars, A. C. M., Kelly, G. A., McNally, A., Simmons, A., Thepaut, J.-J. and Tompkins, A. M.: Analysis and forecast impact of the main humidity observing systems, Tech. Memo., (493), 20, 2006.
Bell, B., Hersbach, H., Simmons, A., Berrisford, P., Dahlgren, P., Horanyi, A., Muñoz-Sabater, J., Nicolas, J., Radu, R., Schepers, D., Soci, C., Bidlot, J., Haimberger, L., Woollen, J., Buontempo, C. & Thepaut, J-N., 2021: The ERA5 Global Reanalysis: Extension to 1950. Quarterly Journal of the Royal Meteorological Society, under review
Dai, A., Wang, J., Thorne, P., Parker, D. E., Haimberger, L., & Wang, X. L. (2011). A new approach to homogenize daily radiosonde humidity data. Journal Of Climate, 24, 965-991. doi:10.1175/2010JCLI3816.1
Dirksen, R. J., Sommer, M., Immler, F. J., Hurst, D. F., Kivi, R. and Vömel, H.: Reference quality upper-air measurements: GRUAN data processing for the Vaisala RS92 radiosonde, Atmos Meas Tech, 7(12), 4463–4490, doi:10.5194/amt-7-4463-2014, 2014. Durre, I., and X. Yin, 2008: Enhanced radiosonde data for studies of vertical structure. Bulletin of the American Meteorological Society, 89, 1257-1262.
Haimberger, L.: Homogenization of Radiosonde Temperature Time Series Using Innovation Statistics, J. Clim., 20(7), 1377–1403, doi:10.1175/JCLI4050.1, 2007.
Haimberger, L., C. Tavolato and S. Sperka, 2008: Towards elimination of the warm bias in historic radiosonde temperature records - some new results from a comprehensive intercomparison of upper air data, J. Climate 21, 4587-4606.
Haimberger, L., Tavolato, C. and Sperka, S.: Homogenization of the Global Radiosonde Temperature Dataset through Combined Comparison with Reanalysis Background Series and Neighboring Stations, J. Clim., 25(23), 8108–8131, doi:10.1175/JCLI-D-11-00668.1, 2012.
Haimberger, L., Blaschek, M., Brönnimann, S., Dahlgren, P., Hersbach, H., Jourdain, S., Laloyaux, P., Sterin, S. and Valente, M. A., 2018: Bias adjustments for global radiosonde data back to 1939. Deliverable 4.2 of EU 7 FP project ERA-CLIM2, 56pp.
Hersbach, H., S. Brönnimann, L. Haimberger, M. Mayer, L. Villiger, J. Comeaux, A. Simmons, D. Dee, S. Jourdain, C. Peubey, P. Poli, N. Rayner, A. Sterin, A. Stickler, M. A. Valente, S. Worley, 2017: The potential value of early (1939-1967) upper-air data in atmospheric climate reanalysis. Quart. J. Roy. Meteorol. Soc.
Hersbach, H, Bell, B, Berrisford, P, Hirahara, S, Horanyi, A, Munoz-Sabater, J, Nicolas, J, Peubey, C, Radu, R, Schepers, D, Simmons, A, Soci, C, Abdalla, S, Abellan, X, Balsamo, G, Bechtold, P, Biavati, G, Bidlot, J, Bonavita, M, De Chiara, G, Dahlgren, P, Dee, D, Diamantakis, M, Dragani, R, Flemming, J, Forbes, R, Fuentes, M, Geer, A, Haimberger, L, Healy, S, J. Hogan, R, Holm, E, Janiskova, M, Keeley, S, Laloyaux, P, Lopez, P, Radnoti, G, de Rosnay, P, Rozum, I, Vamborg, F, Villaume, S & Thepaut, J-N, 2020,: The ERA5 global reanalysis. In: Quarterly Journal of the Royal Meteorological Society. {+}https://doi.org/10.1002/qj.3803+
Imfeld, N., Haimberger, L., Sterin, A., Brugnara, Y., and Brönnimann, S.: Intercomparisons, Error Assessments, and Technical Information on Historical Upper-Air Measurements, Earth Syst. Sci. Data, https://doi.org/10.5194/essd-2020-395, 2021.
Fabio Madonna, Emanuele Tramutola, Souleymane SY, Federico Serva, Monica Proto, Marco Rosoldi, Simone Gagliardi, Francesco Amato, Fabrizio Marra, Alessandro Fassò, Tom Gardiner, and Peter William Thorne, 2020a: The new Radiosounding HARMonization (RHARM) dataset of homogenized radiosounding temperature, humidity and wind profiles with uncertainties. Part I: dataset description and characterisation.
Fabio Madonna, Souleymane SY, Emanuele Tramutola, Federico Serva, Monica Proto, Marco Rosoldi, Simone Gagliardi, Francesco Amato, Fabrizio Marra, Alessandro Fassò, Tom Gardiner, and Peter William Thorne, 2020b: The new Radiosounding HARMonization (RHARM) dataset of homogenized radiosounding temperature, humidity and wind profiles with uncertainties. Part II: comparisons with reanalysis, satellite data and validation of uncertainties.
Nash, J.: Review of test results on the accuracy of radiosonde relative humidity sensors, 2002.
Santer, B. D. and Coauthors, 2005: Amplification of surface temperature trends and variability in the tropical atmosphere. Science, 309, 1551–1556.
Schroeder, S. R., 2008: Adjusting archived radiosonde data using complete validated and inferred radiosonde metadata to compute unbiased atmospheric temperature and moisture trends. Poster Session 4, Climate of the 20th Century (C20C) Poster Session, Wednesday, 23 January 2008, 2:30 PM-4:00 PM, Exhibit Hall B, https://ams.confex.com/ams/88Annual/techprogram/paper_130838.htm (23.11.2020).
Slivinski, L. C., Compo, G. P., Whitaker, J. S., Sardeshmukh, P. D., Giese, B. S., McColl, C., Allan, R., Yin, X., Vose, R., Titchner, H., Kennedy, J., Spencer, L. J., Ashcroft, L., Brönnimann, S., Brunet, M., Camuffo, D., Cornes, R., Cram, T. A., Crouthamel, R., Domínguez‐Castro, F., Freeman, J. E., Gergis, J., Hawkins, E., Jones, P. D., Jourdain, S., Kaplan, A., Kubota, H., Le Blancq, F., Lee, T., Lorrey, A., Luterbacher, J., Maugeri, M., Mock, C. J., Moore, G. K., Przybylak, R., Pudmenzky, C., Reason, C., Slonosky, V. C., Smith, C., Tinz, B., Trewin, B., Valente, M. A., Wang, X. L., Wilkinson, C., Wood, K. and Wyszyński, P. (2019), Towards a more reliable historical reanalysis: Improvements for version 3 of the Twentieth Century Reanalysis system. Q J R Meteorol Soc. 145, 2876-2908. doi:10.1002/qj.3598.
Steiner, A., Ladstädter, F., Randel, W. J., Maycock, A. C., Claud, C., Fu, Q., Gleisner, H., Haimberger, L., Ho, S.-P., Keckhut, P., Leblanc, T., Mears, C., Polvani, L., Santer, B., Schmidt, T., Sofieva, V., Wing, R., and Zou, C.-Z.: Observed temperature changes in the troposphere and stratosphere from 1979 to 2018, J. Climate, 33, 8165–8194, https://doi.org/10.1175/JCLI-D-19-0998.1, 2020.
11. Related articles
