Contributors: N. Clerbaux (Royal Meteorological Institute of Belgium (RMIB)), A. Velazquez Blazquez (RMIB)
Issued by: RMIB/Nicolas Clerbaux
Date: 28/11/2023
Ref: C3S2_D312a_Lot1.2.2.7-v1.0_202303_PQAR_ECVEarthRadiationBudget_v1.1
Official reference number service contract: 2022/C3S2_312a_Lot1_DWD/SC1
History of modifications
List of datasets covered by this document
Related documents
Acronyms
List of tables
List of figures
General definitions
Term | Definition |
Total Solar Irradiance (TSI) | The Total Solar Irradiance (TSI) quantifies the total amount of solar energy that is received by the Earth. It is defined per unit surface perpendicular to the Sun–Earth direction at the mean Sun–Earth distance. The TSI is a fundamental variable governing the climate system, and is recognized as Essential Climate Variable (ECV) by the Global Climate Observing System (GCOS). |
Climate Data Store (CDS) | The front-end and delivery mechanism for data made available through C3S. |
Earth Radiation Budget (ERB) | The difference between the incoming radiant energy to the Earth (directly dependent on the TSI) and the outgoing radiant energy due to reflection and thermal emission. |
Climate Data Record (CDR) | Sufficiently long, accurate and stable time series of a climate variable to be useful to address climate variability and change. |
Interim Climate Data Record (ICDR) | An interim CDRs is an extension of a CDR that meets some timeliness requirements needed in some applications (e.g. “State of the Climate” reports). These preliminary data might not be fully validated and may need to be reprocessed before inclusion in the CDR. |
Fiducial Reference Measurement (FRM) | SI-traceable measurements that provide independent validation of satellite measurements. |
Irradiance | Flux of radiant energy per unit area (usually expressed in W/m² unit). |
Key Performance Indicators | Indicators used to monitor the quality of the C3S services and data quality. |
Bias and bias-corrected Root Mean Squared Difference | The bias (b) is the average value of the difference of the data (xi) with respect to a reference dataset (ri), where N is the number of data points: \[ b = \frac{1}{N} \sum\limits_{i=1}^{N} (x_i - r_i) \]The bias-corrected Root Mean Squared Difference (bcRMSD) is the square root of the average of the square of the differences with respect to the reference dataset, once the bias (b) has been removed friom the data points (xi) (therefore “bias corrected”): \[ bcRMSD = \sqrt{\frac{1}{N} \sum\limits_{i=1}^{N} (x_i - b -r_i)^2} \] |
Scope of the document
This Product Quality Assessment Report (PQAR) summarizes the validation results for the third version (v3.0) of the Climate Data Records (CDR) of the C3S daily Total Solar Irradiance (TSI). The report also covers the first issue (i.e. v3.1) of its temporal continuation as Interim Climate Data Record (ICDR). These records are generated in the frame of the C3S2 312a-lot1 project as a composite of various TSI records.
The scope of the PQAR document is limited to the presentation of the validation results for the CDR and ICDR. The algorithm is presented in a separate Algorithm Theoretical Basis Document (ATBD) [D1] and the methodology followed for the validation is detailed in the Product Quality Assurance Document (PQAD) [D2]. Information about an appropriate use of the TSI data are given in the Product User Guide and Specifications (PUGS) [D3].
Executive summary
The Total Solar Irradiance (TSI) quantifies the amount of solar energy that is received by the Earth, per unit surface perpendicular to the Sun–Earth direction at the mean Sun–Earth distance. The TSI is a fundamental variable governing the climate system and is recognized as an Essential Climate Variable (ECV) by the Global Climate Observing System (GCOS).
Within the Copernicus Climate Change Service (C3S), a long composite Climate Data Record (CDR) is constructed from different TSI measurements obtained by an ensemble of space instruments. To create the composite, the measurements of the individual instruments are first put on a common absolute scale, and their quality is assessed by intercomparison. Then, the composite time series is created as an average of all available measurements, on a daily basis. The full processing is described in an ATBD [D1] and the technical implementation in a System Quality Assurance Document (SQAD) [D4]. The TSI timeseries (CDR and interim ICDR) can be accessed via the Copernicus Climate Data Store (CDS) at https://cds.climate.copernicus.eu.
This document presents the results of the CDR validation. The results of this exercise show that the CDR complies with the target accuracy requirement of 1 W/m². The long term stability seems also to comply with the target requirement of 0.3 W/m²/decade. However, short term variation of the systematic error could slightly exceed this value.
1. Product validation methodology
There are no reference observations (i.e. Fiducial Reference Measurement, FRM) that can be used for a direct validation of the daily Total Solar Irradiance (TSI) record. For this reason, the validation is based on intercomparison with independent TSI records, derived by other teams and based, as far as possible, on independent input data.
The validation methodology is described and justified in the Product Quality Assurance Document (PQAD) [D2]. In short, the validation involves:
The evaluation of the 12 individual input instrument timeseries with the C3S composite and the NOAA NRLTSI2 daily TSI CDR (Coddington et al, 2015) data. This evaluation aims at confirming the accuracy of the adjustment factor, the stability of the input data, and the rejection of some periods and/or specific dates (outliers) for some of the early instruments. Results are presented in Section 2.1.
The intercomparison of the resulting C3S composite CDR with the NRLTSI2 and the SATIRE-S (Yeo et al, 2014a and 2014b) records. These 2 products are the reconstruction of the TSI based on Sun observations like the number of Sunspots. These models are based on independent input data and are then highly valuable for the validation. Results are presented in Section 2.2.
The intercomparison with the Community Consensus TSI record (Dudok de Wit, 2017). This record is however based on mostly the same input data, which limits the usefulness of this record for validation. Results are presented in Section 2.2.
For the Interim CDR (ICDR), i.e. the data after 01.01.2021, a first validation/verification is performed by comparison with the NRTLTSI2 data. A Key Performance Indicator (KPI) is defined that is regularly monitored. Results for the ICDR period (i.e. 01.01.2021 to 31.12.2022 at time of writing) are presented in Section 2.3.
The description of the different reference records used for validation (NRLTSI2, SATIRE-S, Community Consensus TSI) is given in the PQAD [D2].
2. Validation results
2.1 Individual timeseries evaluations
A total of 12 different TSI instruments have been compiled to create the v3.0C3S CDR and v3.1 ICDR, as detailed in the ATBD [D1]. To put them on a same radiometric level, a scaling factor is determined and applied for each instrument. The resulting timeseries are said to be adjusted.
In Figure 2-1, Figure 2-2 and Figure 2-3, each of the adjusted individual timeseries is evaluated by comparison with the C3S composite and the NRL TSI v2 reconstruction. On these Figures, the red parts of the timeseries are the parts that have been discarded as described in the ATBD [D1]. Once the red parts are discarded, the 12 instruments show close agreement with the C3S composite and the NRLTSI2 reconstruction. Table 2-1 provides the bias and the bias-corrected RMS (bc-RMS) of the adjusted individual timeseries with respect to the C3S composite and the NRLTSI2 reconstruction. As expected, the bias between the adjusted individual instrument and the C3S composite is always very small (less than 0.03W/m²). The biases with respect to NRLTSI2 are always positive, and show more variability (between 0.23 and 0.41 W/m²). The RMS difference with the C3S composite is however less informative, as the C3S composite incorporates the different individual instruments under evaluation. The last column, the RMS difference with respect to NRLTSI2, shows a general improvement over time of the quality of the input data (the bias corrected RMS decreases from 0.32 to 0.10 W/m²). However, part of this improvement could be attributed to a better accuracy of the NRLTSI2 reconstruction with time.
Table 2-1: Evaluation of the adjusted individual timeseries with respect to C3S daily TSI composite.
Instrument | Adjustment factor | Number of daily values | Comparison with the C3S composite | Comparison with NRLTSI2 | ||
Bias (W/m²) | bc-RMS (W/m²) | Bias (W/m²) | bc-RMS (W/m²) | |||
ERB | 0.992447 | 3342 | -0.023 | 0.256 | 0.232 | 0.322 |
ACRIM1 | 0.995568 | 2536 | 0.004 | 0.053 | 0.252 | 0.321 |
ERBS | 0.997149 | 4969 | -0.000 | 0.075 | 0.379 | 0.187 |
ACRIM2 | 0.997821 | 3501 | -0.006 | 0.173 | 0.396 | 0.214 |
DIARAD/VIRGO | 0.996449 | 8661 | 0.004 | 0.091 | 0.359 | 0.180 |
PMO06/VIRGO | 1.000181 | 8662 | -0.004 | 0.155 | 0.331 | 0.233 |
ACRIM3 | 1.000078 | 4914 | 0.019 | 0.118 | 0.409 | 0.216 |
TIM/SORCE | 1.000256 | 6000 | 0.008 | 0.073 | 0.352 | 0.112 |
PREMOS | 1.000256 | 1296 | -0.003 | 0.041 | 0.306 | 0.151 |
SOVAP | 0.999345 | 1320 | 0.002 | 0.124 | 0.309 | 0.188 |
TIM/TCTE | 0.999771 | 1858 | -0.023 | 0.050 | 0.286 | 0.140 |
TIM/TSIS1 | 0.999535 | 1086 | -0.026 | 0.022 | 0.251 | 0.101 |
Figure 2-1: Evaluation of the individual daily TSI timeseries for ERB, ACRIM-1, ERBS and ACRIM-2. The green curves show the data used in the C3S composite while the red parts show data discarded from the C3S composite. The blue and orange curves show the ‘running mean’ (121 days time period) of the individual instruments, respectively for the period kept/discarded from the composite. Black and brown curves show the C3S and NRL TSI v2 composites after ‘121 days running mean’.
Figure 2-2: Evaluation of the individual daily TSI timeseries for DIARAD/VIRGO, PMO06/VIRGO, ACRIM3 and TIM/SORCE. The green curves show the data used in the C3S composite while the red parts show data discarded from the C3S composite. The blue and orange curves show the ‘running mean’ (121 days time period) of the individual instruments, respectively for the period kept/discarded from the composite. Black and brown curves show the C3S and NRL TSI v2 composites after ‘121 days running mean’.
Figure 2-3: Evaluation of the individual daily TSI timeseries for PREMOS, SOVAP, TIM/TCTE, and TIM/TSIS1. The green curves show the data used in the C3S composite while the red parts show data discarded from the C3S composite. The blue and orange curves show the ‘running mean’ (121 days time period) of the individual instruments, respectively for the period kept/discarded from the composite. Black and brown curves show the C3S and NRL TSI v2 composites after ‘121 days running mean’.
2.2 Comparison with SATIRE-S, NRLTSI2 and Community Consensus TSI (CCTSI)
In Figure 2-4, the C3S v3.0/v3.1 composite is compared with the NRLTSI2 and SATIRE-S reconstructions, and with the Community Consensus TSI (CCTSI hereafter) composite. the following metrics are estimated to compare the C3S product with respect to NRLTSI2, SATIRE-S and CCTSI:
The average bias (W/m²)
The bias corrected Root Mean Square (bc-RMS) difference (W/m²) for the daily values.
The bias corrected Root Mean Square (bc-RMS) difference (W/m²) after 121-days running means.
Table 2-2 provides these metrics, evaluated over the v3.0 CDR period (01.01.1979 to 31.12.2020). The table also gives the parameters ‘a’ and ‘b’ used to estimate the best fit line, between the bias of the C3S product and the relevant reference dataset, estimated as y = a + b * (x-2001), where the value of 2001 is selected approximately in the middle of the CDR (this choice does not impact the value of b but allows to have intercept value a of the order of the overall bias).
Table 2-2: Bias and bias-corrected Root Mean Square (bc-RMS) differences for the daily values and after 121-days running mean. The ‘a’ and ‘b’ parameters of the linear fit for the bias are given in the last 2 columns.
Reference | Bias (W/m²) | bc-RMS daily values | bc-RMS 121d-mean | Linear fit parameter for the bias y = a + b * (x-2001): | |
(W/m²) | (W/m²) | a (W/m²) | b (W/m²/year) | ||
NRLTSI2 | 0.338 | 0.206 | 0.105 | 0.336 | 0.000017 |
SATIRE-S | 0.293 | 0.198 | 0.178 | 0.287 | 0.012705 |
CC TSI | -0.202 | 0.228 | 0.162 | -0.205 | -0.008601 |
To assess the stability, Figure 2-5 shows the variation between the bias of the C3S composite and the 3 reference datasets (121-days running mean) and the linear fits. With respect to NRLTSI2, the bias is stable with no significant long term trend. On the contrary, there is a clear increase of the bias with respect to SATIRE-S and a clear decrease with respect to the Community Consensus TSI record. Finally, Figure 2-6 shows the temporal variation of the bias corrected RMS difference between the C3S composite and the 3 reference datasets. In this graph, the RMS difference is estimated over a 121-day running windows. The figure shows limited and stable RMS difference with respect to SATIRE-S while there is more variation with respect to NRLTSI2 and the community consensus TSI records.
Figure 2-4: C3S v3.x daily TSI values (grey) and the 121-day running mean (black). The running means for SATIRE-S (red), NRLTSI2 (green) and Community Consensus TSI (blue) are also shown.
Figure 2-5: Bias between the C3S v3.x composite and SATIRE-S (red curve), NRLTSI2 (green curve) and Community Composite TSI (blue curve). The curves shown are 121-day running mean average. The parameters of the linear fits y = a + b (x-2001) where y is the bias and x the decimal year, are given in Table 2-2.
Figure 2-6: bias-corrected RMS differences between the C3S v3.x composite and SATIRE-S (red curve), NRLTSI2 (green curve) and Community Composite TSI (blue curve). The bc-RMS are estimated on a 121-days running mean time period.
2.3 Key Performance Indicator (KPI) for the ICDR product
The v3.1 ICDR period (01.01.2021 - 30.09.2023) is regularly evaluated by intercomparison with the independent NRLTSI2 reconstruction. As Key Performance Indicator (KPI), it is verified that the TSI difference remains statistically consistent with the differences observed over the CDR period (01.01.1979 to 31.12.2020). To this end, the 2.5% and 97.5% percentiles of the difference are estimated over the CDR period, as illustrated in Figure 2-7. These 2.5% and 97.5% percentiles are +0.120 W/m² and +0.537 W/m², respectively. The ICDR verification is performed on the 121-day running mean difference (red curve) and not the daily values (black curve).
The ICDR data from 01.01.2021 to 31.12.2022 remains well within these percentiles.
Figure 2-7: Difference between the C3S v3.0/v3.1 composite and NRLTSI2 daily values (black curve) and 121-days running mean (red curve). Also shown with horizontal lines are the 2.5% and 97.5% percentiles on the 121-days running mean difference (+0.120 and +0.542 W/m², respectively).
2.4 Summary of validation results
The quality assessment of the C3S daily TSI composite v3.0 and its first ICDR extension (v3.1) can be summarized as follows.
The individual time series evaluation (Section 2.1) confirms the values of the adjustment factors and the validity periods determined in the ATBD [D1]. After adjustment, the (absolute) biases with respect to the composite remain well below 0.03 W/m² and the RMS difference with respect to NRLTSI2 goes from about 0.3 W/m² for early instrument (e.g. ERB on Nimbus-7) down to about 0.1 W/m² for the most recent instruments (e.g. TIM).
The comparison with the NRLTSI2 record shows a significant overall bias of 0.34 W/m². This bias is larger than the bias-corrected RMS difference of daily TSI value which is about 0.21 W/m² over the full TCDR period. After a 121-day running mean is calculated, the (max-min) temporal variation of the bias remains limited to 0.6 W/m² over the more than 4 decades covered by the CDR. This is about the double of the GCOS (GCOS-200,2016) stability requirements of 0.01%/decade (i.e. 0.136 W/m²/decade). It is worth mentioning that most of the variation of the bias is observed over the early part of the CDR (01.01.1979 – 31.12.1982) and better stability is observed afterwards, with a (max-min) variation of the level of 0.4 W/m².
The analysis of the differences with SATIRE-S and with the community consensus TSI shows significant temporal variation of the bias, but in opposite directions. For this reason, these two time series are not used for the CDR stability assessment or for the ICDR quality check (KPI).
The results obtained over the ICDR period (1st January 2021 onward) are in general better than over the CDR period. This is explained by the higher quality of the observations from the space instruments which are currently observing the Sun, in particular the TIM instruments. Concerning the KPI, the ICDR period complies mostly with the 2.5% - 97.5% percentiles interval.
3. Application(s) specific assessments
(no application defined yet for this CDR).
4. Compliance with user requirements
Averaged over the full CDR period, the C3S v3.0 daily TSI composite is 0.38 W/m² higher than the NRL TSI v2 reconstruction, 0.29 W/m² higher than SATIRE-S and -0.20 W/m² lower than the community consensus TSI composite. These results indicate that the C3S absolute accuracy complies with the 0.04% (i.e. 0.6 W/m²) accuracy requirement defined by GCOS (2016). The bias is largely dependent on the choice of the reference instrument(s) selected to homogenize the composite timeseries.
From the results presented in this report, it is not possible to conclude that the stability of the C3S TSI record (i.e. the maximum change of the systematic error over a running 10-year time period) complies with the 0.14 W/m²/decade GCOS requirements. The best stability is observed with respect to NRLTSI2 and no significant long-term change is observed over the 1979-2020 time period. However, some significant variations of the bias are observed of the order of 0.6 W/m² (max-min difference) which is about four time the 0.14 W/m²/decade GCOS requirement for stability.
The validation results reported in this document provide evidence that the accuracy is better than 1 W/m² for the daily mean TSI. This accuracy is not constant in time over the duration of the record as the number and the accuracy of the contributing instruments has varied in time.
References
Dudok de Wit, T., Kopp, G., Fröhlich, C., & Schöll, M. (2017): Methodology to create a new total solar irradiance record: Making a composite out of multiple data records. Geophysical Research Letters, 44(3), 1196-1203. https://www.doi.org/10.1002/2016GL071866
Coddington O., J. L. Lean, D. Lindholm, P. Pilewskie, M. Snow, and NOAA CDR Program (2015): NOAA Climate Data Record (CDR) of Total Solar Irradiance (TSI), NRLTSI Version 2. [1984-2020]. NOAA National Centers for Environmental Information. https://doi.org/10.7289/V55B00C1
Available at: https://data.nodc.noaa.gov/cgi-bin/iso?id=gov.noaa.ncdc:C00899
GCOS-200 (2016) : The Global Observing System for Climate Implementation Needs, https://library.wmo.int/idurl/4/55469
Yeo K.L., Krivova N.A., Solanki S.K., Glassmeier K.H. (2014a).Reconstruction of total and spectral solar irradiance from 1974 to 2013 based on KPVT, SoHO/MDI and SDO/HMI observations. Astron. Astrophys. 570, A85 (2014). https://www.doi.org/10.1051/0004-6361/201423628
Yeo K.L., Krivova N.A., Solanki S.K. (2014b) Solar cycle variation in solar irradiance. Space Sci. Rev. (2014). https://www.doi.org/10.1007/s11214-014-0061-7
