Last modified on Sept 08, 2023 16:27

Contributors: H. Konrad (DWD), A. Niedorf (DWD), M. Schröder (DWD), A. C. Mikalsen (DWD), R. Hollmann (DWD), T. Sikorski (DWD), G. Panegrossi (CNR ISAC), E. Cattani (CNR ISAC)

History of modifications

List of datasets covered by this document

Related documents

Acronyms

List of tables

List of figures

General definitions

In the scope of the Copernicus Climate Change Service (C3S), a Climate Data Record (CDR) always has a fixed end point in time, whereas an Interim Climate Data Record (ICDR) is extended continuously, often serving as an extension of a respective completed CDR.
In contrast to this, the record by the Global Precipitation Climatology Project (GPCP) is continuously extended in time, with a latency of two to three months due to the latency of the rain gauge product (see below). In order to avoid confusion, we state here that we will label the GPCP record as CDR in the scope of the brokering to C3S until December 2017, which is the end point for the first delivery to the CDS. Later extensions of the record will be labelled as ICDR in the scope of the brokering to C3S. In this sense, the term 'ICDR' is used here according to C3S terminology.
The GPCP provides an interim product for the monthly solutions, too. It is based on the same satellite data and respective algorithms, but relies on the input of only a preliminary version of the rain-gauge dataset by the Global Precipitation Climatology Centre ('First Guess' in contrast to the 'Monitoring Product' or the 'Full Data Monthly' product). Consequently, it is available much earlier than the fully processed data. When the rain-gauge dataset becomes available and the processing can be completed, the files containing the interim dataset are replaced by the fully processed ones. In the scope of the brokering the GPCP data to C3S, we choose not to broker the interim monthly data in order to avoid confusion and frequent updates. Users interested in these more up-to-date files are referred to the original data repositories.

Scope of the document

This Product Quality Assessment Report summarizes the results of the product assessment of the satellite- and rain gauge-based precipitation estimates by the Global Precipitation Climatology Project (GPCP) at the University of Maryland (UMD). These are brokered to the Climate Data Store (CDS) by the Copernicus Climate Change Service (C3S). The assessment described in this document is carried out within the scope of C3S, whereas the intellectual property rights of the products themselves remain with the GPCP. In this sense, this document is not part of the official GPCP documentation, but produced only in the scope of the brokering to the CDS. The assessment methodology is based on the respective Product Quality Assurance Document [D1].

Executive summary

Estimates of precipitation by the Global Precipitation Climatology Project (GPCP) from the University of Maryland (UMD) are brokered to the Climate Data Store (CDS) by the Copernicus Climate Change Service (C3S). The GPCP is part of the international World Climate Research Programme (WCRP) and its Global Energy and Water Exchanges (GEWEX) project. All intellectual property rights remain with the GPCP. The GPCP precipitation products provided as monthly (v2.3) and daily (v1.3) means are assessed in the context of their inclusion in the CDS.

We perform initial quality (sanity) checks of missing data points and range of the available data points, indicating that the monthly data are complete and the daily data are almost complete. The range of the data is sensible.

A comparison with other gridded datasets (TMPA satellite-based precipitation rates and ERA5 reanalysis data) subject to different extents of averaging (spatial, temporal, none) shows that GPCP precipitation is lower than TMPA in low-latitude oceans, and, in reverse, higher in mid-latitude oceans. Likewise, GPCP precipitation is lower than in ERA5, specifically at low latitudes, but without a respective systematic compensation as in the comparison with TMPA. Overall, the per-grid cell comparison of monthly data exhibits a mean difference of +0.04 mm/d (TMPA) and 0.16 mm/d (ERA5), as well as root-mean-square differences of 1.1-1.2 mm/d. The per-grid cell comparison of daily data is very similar (+0.05 mm/d for TMPA, -0.21 mm/d for ERA5) and five to six-fold increased RMS differences. The uncertainties of the GPCP monthly v2.3 product are briefly assessed in this context, too.

GPCP products have also been compared with the gridded ground-based precipitation radar database NIMROD for Northwest and Central Europe. Over the North Sea and North Atlantic, GPCP tends to have larger precipitation rates compared to NIMROD, but in large parts of continental Europe the GPCP precipitation rates are lower than NIMROD. The differences contain a seasonal cycle with largest absolute difference in summer and lowest in winter. A per-time slice and per-grid cell comparison exhibits a larger mean difference of 0.3 mm/d for the monthly and 0.2 mm/d for the daily dataset, as well as a generally larger variability in the differences than in the comparisons of GPCP to the near-global TMPA and global ERA5 datasets. These more prominent differences could be linked to the variable availability of the ground-based radar data.

The outcome of an additional comparison of GPCP products to in situ data (PACRAIN, OceanRAIN) is constrained by the very different spatial sampling of satellite and in situ observations. Overall, there is good spatially averaged agreement between GPCP and PACRAIN whenever a certain minimum number of PACRAIN stations are available. The comparison with OceanRAIN is only feasible for GPCP daily v1.3. High deviations prevail in this case in a certain fraction of the data, manifesting in an overall RMS difference of ~63 mm/d.

We conclude here with a brief literature review of past assessment efforts with respect to GPCP.

1. Product validation methodology

1.2 Overview

We evaluate the GPCP monthly v2.3 and daily v1.3 precipitation rates according to what was proposed in the related PQAD [D1], i.e. by comparing it to other publicly available datasets. A brief overview of the validation strategy is given in the following. The GPCP monthly product is available on a 2.5° latitude/longitude grid since 01/1979. The spatial resolution of the daily product is 1°, and it is available since 01/10/1996.

We note here that, unless stated otherwise, temporally aggregated values (labelled as ‘daily mean’ and ‘monthly mean’, respectively) are computed consistently with those in the GPCP products. This means that daily values are computed by accumulating sub-daily precipitation over one day and then dividing by one day, resulting in precipitation rates in mm/d¹. For the monthly product, available data are arithmetically averaged over one month. One exception is the OceanRAIN dataset, where the observing instrument moves across GPCP grid cell boundaries, so that we can rarely base a daily OceanRAIN mean in one 1° x 1° grid cell on a full day’s worth of observations (1440 per-minute records). This shortcoming in the GPCP/OceanRAIN comparison goes hand in hand with the very different spatial sampling of satellite data feeding into GPCP and the ship routes in the OceanRAIN database (see below).

For an additional discussion of the products' strengths and limitations, we refer to the respective Target Requirements and Gap Analysis Document [D2], as well as to the original documentation by the GPCP [D3, D4].

1.2 Initial quality checks

As a first measure of quality, we test the number of missing values per file (i.e. per daily/monthly mean field) as well as the mean, minimum and maximum values.

1.3 Comparison with gridded datasets

Here we list first the gridded datasets that we compare the GPCP products to (hereafter labeled as "reference datasets") and their main characteristics. We then give a brief overview of the design of the comparisons that we carry out.

1.3.1 Gridded datasets used for the comparison

TRMM Multi-satellite Precipitation Analysis (TMPA) products are available between 50°S and 50°N from 01/1998 to 12/2019 on a 0.25° equidistant grid as monthly means (product TMPA 3B43 v7; reference TRMM, 2011) and daily means (product TMPA 3B42 v7; reference Goddard Earth Sciences Data and Information Services Center, 2016).

The ERA5 global climate reanalysis (reference C3S, 2017) is available globally over the full GPCP time window at hourly intervals and as monthly means on a 0.25° grid. The hourly ERA5 data are aggregated to daily data (see Section 1.3.2.1) for the comparison with the GPCP daily product. The monthly ERA5 data are used for the comparison with the GPCP monthly product.

Met Office Rain Radar Data from the NIMROD System are available on a 5 km x 5 km grid over parts of Europe in 15-minute intervals since 04/2002 (Met Office, 2003). Because the NIMROD dataset is vastly different from ERA5 and the TMPA products in terms of spatial coverage, we address the comparison with GPCP products separately from these datasets, in Section 2.3. The original NIMROD data are first converted to mm/day and averaged to represent daily means on their native grid. Based on daily means, monthly mean values are computed, again on the native grid. In the case of daily resolution, we filter out daily mean values that are based on less than 32 of quarter-hourly observations (one third in a day). For monthly resolution, we filter out monthly mean values if they come from less than 10 days (~ one third of days in a month) or from less than 1000 quarter-hourly instances (~ one third in a month). For both temporal aggregations (monthly, daily), the data are then brought to the much coarser GPCP grids (2.5° for monthly resolution, 1.0° for daily, see above) by arithmetically averaging all daily or monthly values from the native NIMROD grid cells that fall into respective GPCP grid cells. Respective spatial averages for the GPCP grid cells are discarded if they come from less than 190 NIMROD grid cells in the case of the daily resolution, and 1140, respectively, in the case of the monthly resolution (about one third of the maximum number of NIMROD grid cells in the respective GPCP grid cells). These threshold values are defined ad hoc. They are required because of frequently occurring data gaps in space and time in the NIMROD database. In the Appendix, Supplementary Figure 2 illustrates the regridding scheme and Supplementary Figures 3-7 show how the actual coverage of the data relates to these thresholds. Stronger constraints on data availability during averaging might improve the agreement between NIMROD and GPCP but would also limit the instances of co-existing data pairs in both datasets. Note that the temporal aggregation for the daily resolution NIMROD data differs from that defined for GPCP (accumulation over one day; see Section 1.1).

1.3.2 Methodology

1.3.2.1 Temporal evolution of spatial averages

For each GPCP time slice (daily/monthly), we compute spatial averages over the maximum area available in the respective reference datasets (i.e. between 50°S and 50°N for TMPA products, globally for ERA5, over Europe for NIMROD) for both GPCP and the respective reference datasets. Datasets with higher temporal resolution than GPCP (i.e., ERA5 hourly for comparison with GPCP daily and NIMROD) are averaged temporally to obtain daily and monthly mean values. A regridding has not been carried out at this stage because the spatial integration can be carried out regardless of the specific grids. The resulting time series for GPCP and the reference datasets are then compared. This allows us to also discuss the Key Performance Indicator (KPI) defined for the continuous monitoring of the datasets delivered to the CDS [D6].

1.3.2.2 Climatology

We compare the temporal mean and variability of the GPCP products to the respective reference datasets over overlapping periods. For this, the reference datasets need to be regridded to the 2.5°/1.0° grid of the GPCP monthly/daily products. As all reference datasets are of higher spatial resolution than the GPCP products, the regridding is carried out by averaging the reference datasets in spatial dimensions within each GPCP grid cell. In the case of ERA5 and NIMROD, this is done after the temporal averaging which provides the monthly and daily means addressed in Section 1.3.2.1. The temporal averaging is carried out over the period from 01/1998 to 12/2017 (i.e., the period of TMPA availability). Differences between the temporally averaged two-dimensional fields as well as their respective zonal means are discussed.

1.3.2.3 Single collocated grid cells

Having carried out the temporal (where applicable) and spatial regridding addressed in Sections 1.3.2.1 and 1.3.2.2, we compare GPCP with the regridded reference datasets in each grid cell and at each time in the form of histograms of differences. We also retrieve mean differences and respective root-mean-square (RMS) deviations from the mean for all available grid cells at all times, as well as for five pre-defined categories (zonal, magnitude of the GPCP precipitation rate, elevation, month, and year). The respective comparison is carried out over the full period in which the respective reference datasets are available, i.e. since 1979 for the monthly comparison with ERA5, since 10/1996 for the daily comparison with ERA5, and since 1998 for the daily and monthly comparison with the TMPA products.

We also use the per-time slice and per-grid cell comparison to assess the error budget of the GPCP monthly v2.3 product.

1.4 Comparison with in-situ data

1.4.1 Datasets and Methodology

1.4.1.1 PACRAIN

The Pacific Rainfall Database (PACRAIN, Greene et al., 2008) is a compilation of daily precipitation data at various stations in the Pacific Ocean. Temporal overlap with GPCP exists currently from 1979 to 2015.

Comparing a gridded product with in situ data is often complicated by the spatial or temporal variability at scales below the respective grid resolution. Therefore, for each day or month, we carry out a comparison between PACRAIN based on average values over all stations on the PACRAIN side and over all grid cells in which a PACRAIN station is located and active at the given day or month on the GPCP side. The PACRAIN database consists of daily precipitation values, which are first linearly interpolated to a common daily temporal grid covering 0:00 am to 23:59 pm UTC instead of station-specific 24 hour windows. For the comparison with GPCP daily v1.3, the values from all available stations at a specific day are averaged. For the comparison with GPCP monthly v2.3, all available daily values at each station are averaged to form a station-by-station monthly resolved time series, which is then averaged over all available stations in each month. On the GPCP side, values are averaged over all grid cells in which values from PACRAIN stations are available at that time. Figure 1-1 shows the position of all available PACRAIN stations and the respective GPCP grid cells. The comparison is carried out both for all available PACRAIN stations as well as only for PACRAIN stations situated in atolls and respective GPCP grid cells (distinguished in Figure 1-1). The specific atoll-only comparison is carried out because the presence of topographic effects at the non-atoll-like island stations diminishes the comparability of satellite observations and station data. The additional atoll-only comparison implies a separate averaging over stations and grid cells.

Figure 1-1: PACRAIN stations and GPCP grid cells going into the comparison of PACRAIN and GPCP monthly v2.3 (A) and GPCP daily v1.3 (B). The GPCP grid cells qualify for this comparison if at least one PACRAIN station lies within. At each given time, precipitation rates are averaged only in the available subset of these stations and the respective GPCP grid cells. Note that we distinguish between atoll stations and respective grid cells and the remaining stations and grid cells (see main text). 

1.4.1.2 OceanRAIN

The Ocean Rainfall And Ice-phase precipitation measurement Network (OceanRAIN, Klepp et al., 2017, Klepp, 2015) contains per-minute observations of precipitation rates from ship-borne sensors (rain gauges and disdrometers). For now, we use only the observations based on rain gauges. Data are available from the following ships/periods: Polarstern (2010–2016), Meteor (2014–2016), Investigator (2016–2017), and Roger Revelle (2016).

The routes of these ships along which the rain gauge observations are available, as well as their temporal coverage are displayed in Figure 1-2. Usually, the vessels, as they travel, sample precipitation in one grid cell for less than one day, so that a comparison of OceanRAIN and the GPCP monthly v2.3 product does not make sense. For the comparison with GPCP daily v1.3, we average all hourly rates during one day in one GPCP grid cell. This does not necessarily ensure an adequate spatiotemporal representation of the OceanRAIN in one GPCP grid cell, but where the daily coverage per grid cell is extensive, the comparison provides useful results, see Section 2.4.2.

1.5 Literature review

Comparisons of the GPCP products with other datasets have been published by the GPCP authors and independent ones. We include a brief summary of their findings in section 2.5. However, as the update of the monthly product from v2.2 to v2.3 and the related changes in the daily product (which is dependent on the monthly product) only happened in 2017, the amount of scientific literature covering the new versions is still limited. The literature review makes no claim to completeness.

Figure 1-2: Routes of the four ships from which we use the rain gauge observations here. The routes are plotted in a projection with the centre meridian at 180°E/W (upper panel), and at 0°E/W (lower panel). The panel on the right indicates the temporal coverage by each vessel.

2. Validation results

2.1 Initial quality checks

Figure 2-1 displays the results of the initial quality checks. They indicate that the data in the GPCP monthly v2.3 product are complete and always in an expected range:

• • there are no missing data (Figure 2-1 A, values in red),
  • minimum values are virtually always zero² (Figure 2-1 B, values in red), averages oscillate in a narrow band between 2.49 mm/d and 2.84 mm/d (Figure 2-1 B, values in red), and
  • maximum values oscillate between 16 mm/d and 47 mm/d (Figure 2-1 C), values in red).

Figure 2-1: Number of grid cells without valid precipitation estimates (A), minimum and average values in the two-dimensional precipitation field (B), and respective maximum values (C) for the GPCP monthly v2.3 and daily v1.3 data at a given time (on the x-axis). The scales in A and C are logarithmic, but linear between 0 and 100=1 in A and between 0 and 101=10 in C, for a better illustration of all features in the datasets. The averages here have been obtained as simple averages in the respective precipitation array, without weighting by the area of the grid cells.

The results of the checks on the GPCP daily v1.3 product indicate a stable performance, too. However:

• on 156 days (1.7% of all dates), there are data gaps of variable extent (Figure 2-1 A, values in blue ), peaking at 36,000 grid cells missing which corresponds to ~56% of the spatial field.
• Minimum values are always zero (Figure 2-1 B, values in blue), and the average value oscillates between 2.08 mm/d and 3.48 mm/d (Figure 2-1 B, values in blue).
• Maximum values (Figure 3C, values in blue) oscillate inside a higher range than in the monthly product, between 63 mm/d (5%-percentile) and 132 mm/d (95%-percentile), with a few prominent outliers surging above 1,000 mm/d. Such high values appear to be singular because the respective average values do not follow these surges (Figure 2-1 B, values in blue).

There are a few dates at which all available grid cells in the daily product contain zeros – indicated by zero average and maximum value in Figure 2-1 B and C. These are 04/03/2021, 24/03/2021, 10/06/2021, 19/07/2021, 26/08/2021 and 26/12/2021. These days are also among the dates where the numbers of missing values peak (36,000, Figure 3A), all located at high latitudes, possibly due to the failure of retrieval of the respective polar-orbiting satellite systems [D3, D4]. Therefore, at these dates, the respective fields must be considered as effective data gaps.

The higher variability of both average and maximum values in the case of the daily product, compared to the monthly product (see blue compared with red values in Figure 2-1 B,C), can be understood as a feature of the longer integration time and lower spatial resolution of the monthly product which smooths the temporal evolution and spatial patterns of the precipitation estimates. The very similar centering of average values around ~2.68 mm/d of both the daily and the monthly product is a consequence of their interdependence. As with the higher variability, the higher magnitudes of the maximum values in the daily product are due to the higher spatial and temporal resolution.

The NetCDF files for both the monthly and the daily GPCP product specify a valid range of 0-100 mm/d, which is obviously violated by the daily product (Figure 2-1 C). The recommendation of the GPCP team (personal communication) is to use data even when they lie outside the specified validity range.

2.2 Comparison with gridded datasets (TMPA products and ERA5)

2.2.1 Temporal evolution of spatial averages

Figure 2-2 shows the temporal evolution of spatially averaged (area-weighted) values in the GPCP products and the respective counterparts in the reference datasets ERA5 and the TMPA products (see Section 1.3.1). It is evident that the spatially averaged values of the GPCP products have a far smaller average deviation with respect to the TMPA products than with respect to ERA5. This is not surprising because TMPA 3B43 and GPCP are both calibrated against rain gauge-based data by the Global Precipitation Climatology Centre (GPCC). Note that TMPA products end in 12/2019, while the GPCP and ERA5 time series extend to 12/2021 here.
Table 2-1 contains basic statistics of the differences and shows:

1. that the spatially averaged GPCP values are not significantly biased with respect to TMPA (mean difference exceeded by far by the RMS differences),
2. that the GPCP global means are biased to smaller values with respect to ERA5 (difference < -0.22 mm/d on average in the comparison of monthly data),
3. and that the differences spread up to twice as much for daily means compared to monthly means (RMS column).

Figure 2-2 - A and B: Mean values, averaged over the geographical TRMM window (between 50°S and 50°N) for the GPCP monthly v2.3 (A) and daily v1.3 (B) products and the respective TMPA products (3B43 in A; 3B42 in B). C and D: Global mean values of the GPCP monthly v2.3 (C) and daily v1.3 (D) products and the ERA5 reanalysis, agglomerated as monthly (C) and daily (D) means. E and F: Differences between the spatially averaged values of GPCP and TMPA/ERA5 as shown in panels A–D. The differences are computed between data of the same spatial coverage and temporal resolution, i.e. panel E refers to monthly data and panel F to daily data, and the difference between GPCP and ERA5 is based on global mean values whereas the GPCP/TMPA differences is based on the integration over the ±50° latitudinal window. The vertical dashed black line marks the formal transition from TCDR (until 12/2017) to ICDR (from 01/2018). 

Table 2-1: Basic statistics for the differences between the GPCP and the reference datasets for the temporal coverage of both the TCDR and the ICDR, i.e. the black and blue curves in Figure 2-2 E,F. For the minimum and maximum differences, those GPCP fields that have only zeros and missing values (Section 2.1) and which thus result in a spatial average of zero are discarded. The minimum and maximum differences, the mean, the quantiles (2.5%, median, 97.5%), and the RMS deviations are in mm/d. The RMS is with respect to the mean value. The unit of the slopes is mm/d/decade. In the second to last column, we give the percentage of values that meet the initial target requirement for the KPI accuracy of 0.3 mm/d.

The significantly larger average precipitation in ERA5 compared to GPCP will be analysed in detail in Sections 2.2.2, in which spatial patterns are compared, and 2.2.3, in which the dependence on latitude is briefly discussed.

2.2.1.1 Key Perfomance Indicators

KPIs for TCDR

Table 2-2 contains the same statistics as Table 2-1, but limited to the temporal coverage of the TCDR (i.e. until 12/2017). The initial performance targets for the GPCP TCDR in the scope of the brokering of the data to C3S are 0.3 mm/d for the Key Performance Indicator (KPI) accuracy and 0.034 mm/d/decade for the KPI stability, see e.g. the respective PUGS [D5]. Accuracy in this context is the absolute difference between the spatially averaged value of the evaluated product (GPCP) and a reference product. The second to last column in Table 2-2 gives the percentage of values during the temporal evolution of the spatially averaged values that meet this target. The relatively large difference between mean values in GPCP and ERA5 leads to a much worse compliance with this accuracy target. The comparison with the TMPA products shows that the target is achieved at all times in the case of the monthly product. As discussed already in Section 2.1, the daily products naturally have a larger spread, manifesting in the violation of the 0.3 mm/d accuracy target in ~7.5% of all available days when compared to TMPA 3B43. However, we would expect the compliance with a single target (such as the 0.3 mm/d accuracy target) to be different for the monthly and daily means, so we accept the violation here.

Table 2-2: Basic statistics for the differences between the GPCP and the reference datasets for the temporal coverage of the TCDR (until 12/2017), i.e. the datasets shown as black and blue curves in Figure 2-2 E,F until 12/2017. For the minimum and maximum differences, those GPCP fields that have only zeros and missing values (Section 2.1) and which thus result in a spatial average of zero are discarded. The minimum and maximum differences, the mean, the quantiles (2.5%, median, 97.5%), and the RMS deviations are in mm/d. The RMS is with respect to the mean value. The unit of the slopes is mm/d/decade. In the second to last column, we give the percentage of values that meet the initial target requirement for the KPI accuracy of 0.3 mm/d.

Stability in this context is the absolute slope of a linear regression of the timeseries of differences in spatially averaged values. All absolute values remain below the initial performance target of 0.034 mm/d/dec (last column in Table 2-2).

KPIs for ICDR until 12/2019

The above discussion of KPI achievements is related to the TCDR only, i.e. all data until 12/2017. In compliance with a newly formulated KPI strategy [D6, section 3], we evaluate the ICDR not against a fixed target but against the performance of the TCDR in comparison to the respective TMPA products. As TMPA products were decommissioned during the lifetime of this project, we switched to ERA5 as reference for data after 12/2019 (see section KPIs for ICDR from 01/2020). We test whether the 95% confidence interval of the TCDR differences is valid as 95% confidence interval for the ICDR, too. The boundaries of the 95% confidence interval can be seen in Table 2-2 as:

• the 2.5%-percentile (-0.169 mm/d for monthly resolution, -0.314 mm/d for daily resolution) and
• the 97.5%-percentile (0.185 mm/d for monthly resolution, 0.341 mm/d for daily resolution).

This check on the performance of the ICDR is verified by a binomial test at a 5% significance level, for details see [D6, Table 3]. For the GPCP monthly product, all 24 temporal instances of the ICDR (01/2018–12/2019) fall inside the boundaries, defined by the GPCP TCDR / TMPA comparison. For the GPCP daily product, 720 out of 729 temporal instances fall inside the boundaries. In both cases, we can conclude that the given boundaries are valid as a 95% or higher confidence interval of the ICDR at a significance level of 5%. Consequently, the ICDR in this time period performs sufficiently well, in line with the requirements formulated in [D6, Table 3].

KPIs for ICDR from 01/2020

The decommissioning of TMPA products made it necessary to use ERA5 as the reference dataset for the evaluation of the KPI accuracy for ICDR deliveries covering the period starting in 01/2020. In accordance with our analysis above (e.g., Figure 2-2, Table 2-1, and Table 2-2), we now evaluate globally averaged values instead of being limited to the latitudinal TRMM window. The boundaries of the respective 95% confidence interval in Table 2-2 are:

• the 2.5%-percentile (-0.355 mm/d for monthly resolution, -0.528 mm/d for daily resolution) and
• the 97.5%-percentile (-0.073 mm/d for monthly resolution, +0.013 mm/d for daily resolution).

For the GPCP monthly product, 21 out of 24 temporal instances of the ICDR (01/2020 – 12/2021) fall inside the boundaries defined by the GPCP TCDR / ERA5 comparison. For the GPCP daily product, 702 out of 731 temporal instances fall inside the boundaries. In both cases, we can conclude that the given boundaries are valid as a 95% or higher confidence interval of the ICDR at a significance level of 5%. Consequently, the ICDR in this time period performs sufficiently well, in line with the requirements formulated in [D6].

2.2.2 Climatology

We compare temporal averages of the spatial fields for the common temporal overlap of the three datasets (GPCP, TMPA products, ERA5), i.e. from 01/1998 to 12/2019, which is when the TMPA products were decommissioned. For this, we spatially average the TMPA and ERA5 data to match the coarser GPCP resolutions of 2.5° (monthly) and 1° (daily), see Section 1.3.2.2. TMPA and ERA5 data are already available at the required temporal resolution due to the processing leading to the results in Section 2.2.1.

Figure 2-3 (monthly products) and Figure 2-4 (daily products) show the resulting climatologies (temporal averages since 1998) and the temporal variability for GPCP, TMPA, and ERA5 on the respective GPCP grids, as well as the differences between GPCP and TMPA/ERA5 products. The dominant pattern is higher rain rates in the tropics compared with other regions in all three datasets at both temporal resolutions. The temporal variability in a grid cell is in general positively correlated to the magnitude of the temporal mean, i.e. high temporal variability in areas of high precipitation. As expected, the temporal mean is similar for monthly (Figure 2-3) and daily (Figure 2-4) data. However, the temporal variability does in general increase in the daily product for GPCP, TMPA, and ERA5, due to added temporal (sub-monthly) and spatial (1.0° vs. 2.5° resolution) variability.

Differences between the GPCP and the TMPA products (Figure 2-3 G and Figure 2-4 G) are similar for monthly and daily means. The differences over land are not very prominent as expected due to the calibration to GPCC data. Over the oceans, GPCP tends to have higher precipitation rates than the TMPA products at mid-latitudes (~1 mm/d) and vice versa in the tropics, reaching stronger deviations between the two datasets here in places.

Figure 2-3 – A-F: Climatologies (i.e. temporal mean) for GPCP monthly v2.3, TMPA 3B43, and ERA5 (left column), and the respective standard deviation as a measure for the temporal variability (right column). Data are averaged over 01/1998 to 09/2019 (i.e., the period of TMPA data availability). Panel G shows the difference between the GPCP and the TMPA 3B43 climatology ("A minus C"), and panel H shows the same for the GPCP/ERA5 comparison. This implies that negative values in G and H occur where GPCP has lower precipitation rates than the respective reference dataset, and vice versa for positive values.

Figure 2-4: The same as Figure 2-3, but for daily products. 

Differences between the GPCP and ERA5 at the two different temporal resolutions are also very similar (Figure 2-3 H and Figure 2-4 H). Here, it is evident that the higher global mean values in ERA5 compared to GPCP (Section 2.2.1) originate in the tropics, where precipitation is often more than 1 mm/d higher in ERA5 than in GPCP. Other areas of prominent mismatch between GPCP and ERA5 exist, both over land and ocean, such as higher precipitation in ERA5 in the Himalaya, along the East Antarctic coast, and the North and South American Pacific coasts, and lower precipitation in ERA5 in parts of the Southern Ocean (specifically the South Atlantic), parts of the Congo and Amazon catchments, and bits of the Malay Archipelago. Comparison of the zonal mean climatology (Figure 2-5) shows the same prominent shift to higher rates in the tropics in the case of ERA5, and an unevenly oscillating relation between ERA5 and GPCP in the mid- and high-latitudes.

The zonal means (Figure 2-5) confirm that GPCP products yield higher precipitation rates than TMPA at mid-latitudes, specifically in the Southern hemisphere, whereas in the tropics, this relation is reversed.

Figure 2-5 – A: Zonal means of the climatologies (Figure 2-3 and Figure 2-4). In the absence of inherent differences between monthly and daily aggregations in each product, these should have the same magnitudes here, if they were evaluated on a common grid. However, as all data here are regridded to the respective GPCP grids (2.5° for monthly data and 1.0° for daily data), differences here are due to averaging over coarser/finer spatial scales for monthly/daily data. B: Differences between GPCP products and the respective reference datasets (to which the color codes refer, see legend in panel A). Negative values indicate that GPCP has lower precipitation rates than the respective reference dataset, and vice versa for positive values.

2.2.3 Single collocated grid cells

Having the four reference datasets (TMPA, ERA5 as monthly and daily means) on a grid that matches the grids of the two GPCP products in spatial and temporal dimensions, we perform a comparison per grid cell and time slice. Respective time periods are 01/1979–12/2019 for the comparison of the GPCP monthly product against ERA5, 10/1996–12/2019 for the comparison of the GPCP daily product against ERA5, and 01/1998–12/2019 for the comparison of both GPCP products against respective TMPA products. The results are summarized in Figure 2-6 and Table 2-3. Again, it is visible that the daily products have a larger spread between them, manifesting in much larger RMS deviations. The mean difference to TMPA tends to be slightly positive. The mean difference to ERA5 is negative and of slightly higher magnitude, due to less precipitation in the tropics in the GPCP products (see Section 2.2.2).

Table 2-3: Number of per time slice and per grid cell comparisons shown in the histograms of Figure 2-6, as well as the mean and RMS deviation (in mm/d each).

Figure 2-6: Histograms of differences between the GPCP datasets and the respective reference datasets. The differences are computed for every available time step and every GPCP grid cell. Table 2-3 lists statistics of these distributions. The histogram bins are 0.5 mm/d wide. 

The differences between the GPCP and reference datasets are, of course, not random but depend on a variety of factors. Supplementary Figure 1 in the Appendix displays such dependencies, for example a strong latitudinal dependency at least of the RMS deviation, which tends to decrease towards the poles (Supplementary Figure 1, first column). A trend from negative to positive or at least less negative mean values with increasing (absolute) latitude is also visible and complies with the findings in Section 2.2.2 (Figure 2-5). The decreasing RMS deviation towards the poles is probably at least partially related to less precipitation at higher latitudes, and the fact that GPCP agrees with the comparing datasets better at low precipitation (Supplementary Figure 1, second column, indicating runaway differences for higher GPCP precipitation rates; however also occurring in much fewer instances). The same is probably true for smaller differences at higher elevation (Supplementary Figure 1, third column), because precipitation tends to level off at high altitude. A certain seasonality in the differences between GPCP and ERA5 can be detected for the Northern mid-to high latitudes (Supplementary Figure 1, fourth column), and less so for the southern equivalent and the low to-mid-latitudes. The mean and RMS deviation of the differences has not changed over time (Supplementary Figure 1, last column).

Note that any deviation in the mean difference of GPCP and the two reference datasets at monthly and daily resolutions (Table 2-3 and Figure 2-6) does not need to stem from different periods here, as in Section 2.2.1, but can also be due to the varying spatial resolution of 1.0° and 2.5°, respectively.

2.2.3.1 Error assessment

The GPCP monthly v2.3 product comes with an uncertainty estimate that we assess in the following. No such uncertainty information is provided with the GPCP daily v1.3 product.

For the monthly-resolved datasets and the time period 01/1979–12/2019 (TMPA availability), we compute the absolute normalized difference between GPCP and the respective reference datasets as the ratio of the difference between GPCP and the respective reference dataset (∆pijk) at each available time slice k, each latitude grid node i, and each longitude node j as evaluated in the context of the collocated grid-cell comparison above and the GPCP uncertainty σ_ijk:

A value of 1 or less in Equation (1) indicates that the difference between GPCP and the comparing dataset is within the GPCP error budget σ_ijk, see also Immler et al. (2010), Equation (6).

Figure 2-7: Cumulative fraction of values for which the absolute normalized difference in precipitation, see Equation (1), is at or below a given number (x-axis).

Figure 2-7 shows the respective cumulative distribution, i.e. for every absolute normalized difference value  along the x-axis, the respective number of values according to Equation (1), for which  is given as percent of the total number of values , see Table 2-3. About two thirds of the data points (~ 1 sigma) are at or below an absolute normalized difference of 1. Note that we do not consider the uncertainty of the reference here. This would increase the observed fraction, indicating that the provided GPCP uncertainty is larger than the one-sigma interval [D6, based on Immler et al. 2010].

2.3 Comparison with gridded datasets (NIMROD)

We discuss differences between the GPCP products and precipitation estimates derived from ground-based precipitation radar stations from the Met Office's NIMROD database between 2002 and 2019. These are available over central Europe and the British Isles (Figure 2-8), covered by a maximum of 56 grid cells in the GPCP monthly product and 349 in the daily product.

Figure 2-8:  Climatologies (temporal mean over the period 2002-2019) in the monthly and daily representations of GPCP and NIMROD, over full years and per season, in the case of the monthly products. The first and second column show the temporal means for GPCP (panels A, B; top row) and NIMROD (panels G, H; second row) and the respective difference (GPCP minus NIMROD, panels M, N; third row) for full annual cycles, for the monthly and daily data, respectively. The remaining four columns show seasonal averages and differences over the period for the monthly comparison (mam=March, April, May; jja=June, July, August; son=September, October, November; djf=December, January, February). A grid cell at a given time only contributes to the averages if it is available in both the GPCP and the NIMROD datasets, manifesting in a varying number of months or days over which the values are averaged (panels S through X, bottom row).

Table 2-4: Results of the per-time slice and per-grid cell comparison of GPCP and NIMROD. Listed are the numbers of available data pairs (i.e. grid cells and times for which GPCP and NIMROD are available simultaneously), as well as the mean, 2.5- and 97.5-percentiles, RMS deviation, and – in the case of the monthly product – mean GPCP uncertainty (in mm/d each).

Figure 2-9: Temporal evolution of spatially averaged values from the GPCP and NIMROD datasets, and respective differences, for the monthly (A) and daily (B) products. Similar to the temporal means (Figure 2-8), a grid cell only contributes to the spatial average at a given time, if both the GPCP and the NIMROD product are available at that time in that grid cell (see the varying number of contributing grid cells at the bottom of each sub-figure).

In the climatological analysis (comparable to Section 2.2.2 about the comparison of climatologies for GPCP, TMPA products, and ERA5), general patterns are captured similarly by both datasets, such as higher rates of precipitation over parts of the British Isles and in or around the Alps (Figure 2-8A, B, G, H). However, differences are also clearly visible. For example, the land masses tend to stand out as negative (GPCP precipitation < NIMROD precipitation) at the finer resolution of the daily product comparison (Figure 2-8 N), whereas over the North Sea, Irish Sea and Bay of Biscaya, the situation is reverse. Especially at the boundaries of the area covered by the radar stations going into the NIMROD dataset, this mismatch might well originate from the decrease in sensitivity of ground-based radar systems through energy attenuation at greater distances from the stations.

The time series of spatially averaged precipitation in the GPCP and NIMROD datasets for monthly and daily resolutions are shown in
Figure 2-9 (comparable to Section 2.2.1 about the GPCP–TMPA/ERA5 comparison). Their differences do mostly level around zero, but the temporal variations are large, containing also a very pronounced annual cycle and possibly a trend in the time before 2011. The oscillation in the differences is strongest before 2006 when amplitudes of this cycle decrease, possibly due to changes in the GPCP or NIMROD observing system. The annual cycle is also visible in the per-season climatologies (Figure 2-8): GPCP tends to see lower precipitation rates in large parts of continental Europe than NIMROD in spring and summer (panels O and P), whereas this tendency is not as prominent in autumn (panel Q) and absent in winter (panel R). The trend that is visible in the GPCP-NIMROD differences in the early period (before 2011) in Figure 2-9 could originate in the NIMROD dataset, as this product shows an apparent trend and oscillations on its own in the period (Figure 2-9 A, blue line), that could be due to a trend towards more observations (panels A in Supplementary Figures 3-7).
Possible explanations for the occurrence of this seasonal cycle in the mismatch of the datasets include:

• The capability of satellite-borne radiometers for mapping convective precipitation is much diminished compared to other forms of precipitation, due to their short-lived and spatially discontinuous occurrence. The better-resolved ground-based radar observations are less likely to miss such events. Consequently, the higher frequency of such precipitation during the summer months may lead to the observed seasonal pattern.
• A hypothetical seasonal cycle in the typical daily precipitation pattern would be sampled differently by polar-orbiting satellites and the 24/7 monitoring of the ground-based radar stations.
• For both passive satellite observations and ground-based, active radar observations, the observed signals depend on the precipitation type (rain, snow, hail, etc), which might cause a seasonal cycle in the mismatch of the two datasets.
• GPCP datasets over land are calibrated against the GPCC rain-gauge based product; consequently, mismatches between GPCP and NIMROD in central Europe and their seasonal cycle may already be present in a respective GPCC–NIMROD comparison.

The alteration of the difference between GPCP and NIMROD around zero (spatially averaged values as black lines in
Figure 2-9 A, B) is confirmed by the relatively low mean difference in the grid cell-wise comparison (~0.3 mm/d; Table 2-4 similar to the GPCP–TMPA/ERA5 comparison in Section 2.2.3), less than the average GPCP monthly uncertainty (0.4 mm/d; Table 2-4). However, the spatio-temporal variation around zero is large, manifesting in high absolute values for the given percentiles (Table 2-4), as well as for RMS deviation (2.5 mm/d for the monthly GPCP–NIMROD comparison (Table 2-4) vs. ~1.2 mm/d for the GPCP–TMPA/ERA5 comparison (Table 2-3); 8.2 mm/d for the daily GPCP–NIMROD comparison vs. ~6 mm/d for the GPCP–TMPA/ERA5 comparison).

2.4 Comparison with in-situ data

2.4.1 Comparison with PACRAIN

Due to the lack of transferability of point measurements such as in the PACRAIN dataset and a relatively coarsely gridded product as GPCP (see Section 1.4.1), the comparison with PACRAIN is carried out on the basis of averages over all available stations illustrated in Figure 1-1 at a given time (PACRAIN) and the average over all grid cells in which PACRAIN stations are available (GPCP). An additional brief analysis is carried out for averages over all atoll-like settings and respective GPCP grid cells.

Figure 2-10 A and Figure 2-11 A display the temporal evolution of the averages over all stations and respective GPCP grid cells, as wells as the respective differences. A step from a clearly negative bias of about 2 mm/d to almost zero occurs in 10/2002, when many more stations become available in the PACRAIN dataset. At the same time, the number of GPCP grid cells did not change as much (Figure 2-10 C and Figure 2-11 C), so that we conclude that the PACRAIN average became much more representative of the regional mean at that time (for which the GPCP average should be representative, too), due to the improved spatial sampling. The histograms in Figure 2-10 B,D and Figure 2-11 B,D show that the overall differences (panels B) are more skewed towards negative values than the ones filtered with respect to a minimum number of PACRAIN stations available (i.e. data from 10/2002; panels D). This feature is also visible in the statistics of the respective histograms, Table 2-5, where the mean difference is closer to zero if the data are filtered with respect to the number of PACRAIN stations.

Figure 2-10 – A: Time series of monthly precipitation averaged over all available PACRAIN stations in each month (blue), as well as over all GPCP monthly v2.3 grid cells in which PACRAIN stations were available in each month (red). The black line shows the difference between these two products. Negative values indicate that the PACRAIN average precipitation rate is higher than the GPCP average rate. B: Histogram of differences between GPCP average values and PACRAIN average values as shown in the black graph in A. C: Number of PACRAIN stations available in each month and number of GPCP grid cells in which these PACRAIN stations were available. D: Same as panel B, but with at least 120 PACRAIN stations available. For statistics of the histograms in B and D, see Table 2-5. 

Figure 2-11: Same as Figure 2-10, but for the GPCP daily v1.3 and daily means of the PACRAIN stations. Panel D is the same as panel B but for more than 100 PACRAIN stations being available (in contrast to 120 in Figure 2-10). For statistics of the histograms in B and D, see Table 2-5. 

Given the lack of representativeness at PACRAIN station numbers below 120 or 100 (monthly or daily, respectively), we discuss here only the comparison of data where the number of stations is above 120/100, i.e. Figure 2-10 D and Figure 2-11 D. The monthly product captures the regional mean in the Pacific Ocean with a mean difference of only 0.1 mm/d and a respective RMS deviation of only 1.6 mm/d (Table 2-5). This is approximately at the same magnitude as the average GPCP uncertainty (Table 2-5), indicating that the data in the GPCP monthly product can be considered a good representation of the regional precipitation in the Pacific Ocean.

The spread in the daily product is again larger than in the monthly product, manifesting in a RMS deviation of 3.6 mm/d (Table 2-5), due to the lesser degree of temporal (and in the case of GPCP also spatial) smoothing. A comparison between mean difference and RMS deviation and a respective average regional uncertainty cannot be carried out because no uncertainties are provided with the daily GPCP product.

Mountainous topography affects precipitation patterns strongly, which is represented in station data but cannot be resolved by satellite-based observations. Earlier comparisons between GPCP products and PACRAIN data have therefore been restricted to those stations in the PACRAIN database that are situated in atolls (e.g. Pfeifroth et al., 2013). The 'atoll-labelled' rows in Table 2-5 indicate that the agreement for these sites is slightly better (smaller absolute extreme differences, similar mean difference, slightly smaller RMS deviation) for monthly resolution. However, at daily resolution the agreement at atoll stations is actually diminished.

Table 2-5: Statistics of the GPCP/PACRAIN comparison as shown in Figure 2-10 and Figure 2-11 (rows labelled as 'all'). All values are differences between the respective GPCP product, averaged over all grid cells in which PACRAIN stations are available at a given time, and the respective PACRAIN average over all available stations at a given time. Minimum, maximum, and mean values as well as the RMS deviation from the mean value and the mean GPCP uncertainty are in mm/d. Also included are rows where the statistics are given for the subset of temporal instances where a minimum number of PACRAIN stations is exceeded, and for the separate atoll-only comparison where only the subset of PACRAIN stations situated in atolls and the respective GPCP grid cells have been averaged for each time step. Note that these latter '> n PACRAIN stations' rows (with n=100, n=120) provide the statistics to a sub-set of the timeseries to which the 'all' rows provide respective statistics (i.e., filtering out specific points in time, depending on overall station availability), whereas the 'atoll stations only' provide statistics to the separate timeseries that is compiled by averaging over a smaller amount of stations and GPCP grid cells in the first place (i.e., filtering out specific stations/grid cells). Consequently, it is not contradictory that the numbers of values are equal in the 'all' and 'atoll stations only' rows, and that minimum/maximum values are lower/higher in the 'atoll stations only' rows.

2.4.2 Comparison with OceanRAIN

The comparison with OceanRAIN observations is carried out only for GPCP daily v1.3 and the rain gauge observations onboard the research vessels (RV) Polarstern, Meteor, Investigator, and Roger Revelle (see Section 1.4.1.2 and Figure 1-2). Figure 2-12 A and B show a large spread of differences for the complete dataset, around a mean of -4.9 mm/d (i.e. higher rates in the OceanRAIN dataset) and a RMS deviation of 62.9 mm/d. Figure 2-12 C shows that this large spread is dependent on many factors. For example, mean and RMS deviation are -0.4 mm/d and 14.3 mm/d, respectively, for latitudes below 36.5°, and, in contrast, -15.6 mm/d and 105.0 mm/d for latitudes above 77.5°. This manifests also in a much smaller spread for RV Meteor and RV Investigator which sample less polar areas than RV Polarstern. Albeit at very low latitudes, the observations by RV Roger Revelle are only few (133 daily per-grid cell means in total), so that the higher spread can be explained by the small size of the sample.

The number of OceanRAIN observations in one GPCP grid cell in one day has a strong correlation with the spread of GPCP/OceanRAIN differences, too. The RMS deviation decreases from 113.2 mm/d for 68 observations or less to 19.4 mm/d for 329 observations or more (Figure 2-12 C, n_obs-category). The GPCP grid cells covered by more than 1000 per-minute OceanRAIN observations (569 in total) yield a RMS deviation from the mean difference (-0.2 mm/d) of only 5.7 mm/d (not shown in Figure 2-12).

The relatively pronounced temporal changes in the mean and RMS difference (Figure 2-12 C, time category) are probably also linked to the routes of the vessels and thus their sampled latitudes.

While the RMS deviations between GPCP and OceanRAIN are in general quite large (between 7.5 mm/d and 114.5 mm/d in the categories in Figure 2-12 C, and 63.1 mm/d for the whole data set; see category 'all'), it should be noted that these high magnitudes are governed by only a certain fraction of extreme values. For example, visual inspection of the histogram in Figure 2-12 B leads to the conclusion that the high overall RMS value (63.1 mm/d) must stem from few but notably strong outliers. Indeed, about 57.4% of all GPCP/OceanRAIN absolute differences in Figure 2-12 B remain below 1 mm/d, 83.9% remain below 5 mm/d, and 92.1% remain below 10 mm/d.

We note here that a comparison between the per-minute OceanRAIN observations and the instantaneous pixel-wise precipitation rates from the involved satellite sensors (in contrast to the daily gridded mean of the final product) would be more feasible, see e.g. Burdanowitz et al. (2018) for a respective comparison of OceanRAIN observations with a different satellite product, the Hamburg Ocean Atmosphere Parameters and Fluxes from Satellite Data (HOAPS 4.0). It is, however, not an option for us because we do not have access to Level 2 intermediate data of the GPCP. Finally, it is noted that rain gauges onboard ships are not well suited for precipitation measurements (Klepp et al., 2018), visible for example in the strong negative bias of ca. -15 mm/d at latitudes above 77.5° (Figure 2-12 C), where OceanRAIN observations indicate precipitation rates that are unrealistically high in the cold Arctic or Antarctic climate. Thus, future efforts will focus on OceanRAIN disdrometer data.

Figure 2-12 – A: Time series of differences between GPCP daily v1.3 and daily mean values in respective GPCP grid cells based on OceanRAIN observations. The different colors represent the different vessels, see x-axis annotations in panel C (ship category) and Figure 1-2. Note that for each day and ship there can in principle be more than one GPCP grid cell populated with OceanRAIN observations. B: Histogram of all available differences. C: Mean (red circles, left y-axis) and RMS deviation (black diamonds, left y-axis) of differences in various categories. The categories are 'all', vessels, timing in the OceanRAIN period (2010-2017), zonal, number of OceanRAIN observations (nobs) per minute in daily GPCP grid cell, and GPCP precipitation magnitude (p_GPCP). (Note: panel B is based on the category 'all') is the one shown in panel B. The blue crosses in panel C indicate number of daily per-grid cell differences in the respective categories (right y-axis, logarithmic). The limits of the bins in numerical categories (time, zonal, nobs, p_GPCP) have mostly been designed to contain a similar number of values.

2.5 Literature review

As GPCP is one of the most used precipitation products³, earlier versions of GPCP have been compared to other products regularly. Many efforts have involved the comparison of GPCP to the same reference datasets as deployed here, or earlier versions of this, such as ERA-Interim instead of ERA5 (e.g. Bosilovich et al., 2008; Szczypta et al., 2011), earlier TMPA releases (e.g. Dinku et al., 2007; Joshi et al., 2013) or shorter versions of the PACRAIN database (Pfeifroth et al., 2013). Often, such comparisons have focused on certain regions rather than on the planet as a whole (e.g. McPhee and Margulies, 2005; Bolvin et al., 2009; You et al., 2015). Some findings discussed in this PQAR have been featured in these earlier publications, for example the reasonable correlation between GPCP and PACRAIN (Pfeifroth et al., 2013), or the higher tropical precipitation rates in ERA products (Bosilovich et al., 2008).

The newly released versions of GPCP have not been available long enough for a similarly extensive publication base. Publications that explicitly evaluate the v2.3 monthly or the v1.3 daily product are discussed below.

Adler et al. (2018) describe the latest version of the dataset. Adler et al. (2017) find that average GPCP and TRMM-based rates over the tropical oceans compare well to each other. Based on satellite-gravimetric observations, Behrangi et al. (2018) show that GPCP monthly v2.3 represents Northern high-latitude precipitation better than gauge-only products which feature a strong bias where solid precipitation is not adequately represented by an in situ measurement. Sun et al. (2018) compare GPCP precipitation rates to various other datasets. Wang et al. (2018) use GPCP monthly v2.3 to assess different IMERG-based precipitation rates globally. Masunaga et al. (2019) investigate extreme values in global precipitation in various datasets including GPCP products and find that relations between the mean values from different datasets are vastly different from relations between extreme values from different datasets.

3. Application(s) specific assessments

In addition to the extensive product validation (see chapter 2 for results and chapter 2/3 in [D1] for validation methodology) a second assessment is introduced to evaluate the Interim Climate Data Record (ICDR) against the Thematic Climate Data Record (TCDR) in terms of consistency. Since frequent ICDR deliveries make detailed validation not feasible, a consistency check against the deeply validated TCDR is used as an indication of quality. This is done by a comparison of the following two evaluations:

• TCDR against a stable, long-term and independent reference dataset
• ICDR against the same stable, long-term and independent reference dataset

The evaluation method is generated to detect differences in the ICDR performance in a quantitative, binary way with so called Key Performance Indicators. The general method is outlined in [D6] chapter 3. The same difference between TCDR/ICDR and the reference dataset would lead to the conclusion that TCDR and ICDR have the same quality (key performance is "good"). Variations or trends in the differences (TCDR/ICDR against reference) would require a further investigation to analyze the reasons. The key performance would be marked as "bad". The binary decision whether the key performance is good or bad is made in a statistical way by a hypotheses test (binomial test). Based on the TCDR/reference comparison (global means, monthly or daily means) a range is defined with 95% of the differences are within. This range (2.5 and 97.5 percentile) is used for the ICDR/reference comparison to check whether the values are in or out of the range. The results could be the following:

• All or a sufficient high number of ICDR/reference differences lies within the range defined by the TCDR/reference comparison: Key performance of the ICDR is "good"
• A smaller number of ICDR/reference differences is within the pre-defined range: Key performance of the ICDR is "bad"

3.1 Results

The results of the KPI test are summarized in Table 3-1.

Table 3-1: Summary of KPI results with 2.5 and 97.5 percentiles and number of ICDR months within the range. Colors green or red mark the results of the binomial tests as good or bad, respectively.

Percentiles were calculated based on the comparison of the TCDR of the GPCP dataset against TRMM TMPA data as reference dataset (for the ICDR of 2019) and ERA5 (from 2020 on). The switch became necessary since the TMPA data was no longer available. Most of the ICDR months/days are within the pre-defined percentiles and lead to "good" KPI tests. The absolute difference between GPCP and ERA5 data is slowly increasing identifying ERA5 as the reason for this. This trend could lead to another switch of the reference dataset since the percentiles are based on the TCDR-ERA5 comparison and wouldn't be met permanently.

4. Compliance with user requirements

There are no specific user requirements published for GPCP, to our knowledge.

 Target requirements within the present framework of brokering GPCP v2.3 monthly and v1.3 daily data to the CDS were formulated for global mean values. With TMPA products serving as reference datasets for the TCDR part of the datasets (until 12/2017) and for the ICDR part between 01/2018 and 12/2019, we adapt this requirement to the TRMM window (inside ±50° latitude). For data from 01/2020, ERA5 serves as reference dataset, so that global mean values are evaluated henceforth. The details of the analysis can be found in Section 2.2.1.1. The methodology, especially for the analysis of the accuracy of the ICDR part, is outlined in the KPI document [D6, section 3].

 For the TCDR part of the datasets (until 12/2017), the accuracy, i.e., the differences between global mean values of the GPCP datasets and of a defined reference dataset, is required to remain inside ±0.3 mm/d. This requirement is met by the GPCP monthly dataset with the respective monthly TMPA product as reference. It is violated by the GPCP daily dataset with the respective daily TMPA product as reference in ~7.5% of all daily instances. However, daily and monthly data were unlikely to meet the same fixed target requirement initially designed for monthly values, due to the higher temporal variability in daily data. The minor violation is acceptable in this light.

 For the TCDR part, the requirement for the stability of the dataset, i.e., the linear trend in time of the differences of spatial averages (see paragraph above) is to remain below 0.034 mm/d/dec. Both monthly and daily GPCP data meet this requirement with respective TMPA products as references.

 For the ICDR part of the dataset (from 01/2018), the requirement for the accuracy is derived from the respective performance of the TCDR. The 2.5%- and 97.5%-quantiles in the differences of (near-) global averages between GPCP datasets and the respective reference datasets are formulated as target requirements. We count in how many instances (days or months, respectively) the differences of spatial averages fall inside these two TCDR-based quantiles, i.e., inside the 95% confidence interval of the TCDR. A binomial test is carried out on the number of ICDR outliers vs. the number of total ICDR instances at a significance level of 5%. The dataset is considered compliant with the target requirements if the test verifies that the two specified quantiles, i.e., the position of the 95% confidence interval of the TCDR, are upheld for the ICDR. Both the daily and the monthly GPCP datasets comply with these requirements. Note that the reference datasets changed from respective TMPA products to ERA5 when the former were decommissioned in 12/2019.

 No requirement for the stability is formulated for the ICDR part of the dataset, as the retrieval of a temporal trend on such short time scales (less than 4 years at the end of the current brokering commitment) would most likely be spurious.

Appendix

Supplementary Figure 1 (overleaf): Mean and RMS deviation in the per grid cell and time slice comparison for (column-wise) varying categories (latitude of the centre of the grid cell, GPCP precipitation rate, elevation, seasonality, and year) and (row-wise) the comparisons to the four reference datasets (GPCP vs TMPA/ERA5 monthly/daily). The panels show mean (red circles) and RMS deviation (black diamonds) on the precipitation difference scale (left axis in each panel; annotation on left-column panels), depending on whether a given grid cell in a given time slice falls into the respective categorical bin (for example a certain latitudinal band in the 'latitude' category). For each category and bin, we also give the number of data (blue crosses, right axis in each panel, logarithmic scale given in the panels on the right). Note that elevation ≤0 indicates oceans. Elevation exceeding 4000 m is included in the bin centered at 4000 m. Months are: 1-12: Jan to Dec at latitudes above 50°N; 13-24: Jan to Dec at latitudes between 50°S and 50°N; 25-36: Jan to Dec at latitudes below 50°S. In each category, the numbers of all comparisons given by the blue crosses add up to the total number as given in Table 3. 

Supplementary Figure 2: regridding scheme for bringing the NIMROD observations to the respective GPCP spatiotemporal grids. The respective thresholds, below which data are discarded, and the Supplementary Figures (S.F.), in which their relations to the actual data volumes are shown, are given in the arrows that indicate the workflow.

Supplementary Figure 3:  Daily coverage in the NIMROD dataset on the original European-wide grid; panel A shows the temporal evolution of maximum and average number of 15-minute intervals for which precipitation data are available on each day in the NIMROD database over the entire covered region. Note that for the average values, we filter out NIMROD grid cells that contain zero observations over the entire time span (grey in panel B). The horizontal black line indicates the threshold value 32, below which we discard data in a given grid cell on a given day (see Section 1.3.1). Panel B shows the average number of 15-minute intervals available in each of the original grid cells. Note that we filter out days that contain zero data. 

Supplementary Figure 4: Same as Supplementary Figure 3, but for 15-minute intervals over all months. Consequently, the numbers are higher, and the threshold in panel A is 1000, see Section 1.3.1. Note that for the filtering of monthly data, the number of days in a month that are covered by observations (Supplementary Figure 5) is considered as well (Section 1.3.1). 

Supplementary Figure 5: Same as Supplementary Figures 3 and 4, but for days in a month covered by when the threshold of 32 15-minute intervals in one day is passed. Here, the respective threshold below which data are discarded is 10 days, see Section 1.3.1. Note that for the filtering of monthly data, the numbers of 15-minute intervals in one month that are covered by observations (Supplementary Figure 4) are considered as well (Section 1.3.1). 

Supplementary Figure 6: Same as Supplementary Figures 3-5, but for NIMROD grid cells contained in the 1° GPCP grid cells for the daily averaged product (i.e., based on the data availability outlined in Supplementary Figure 3). Here, the respective threshold below which data are discarded is 190 NIMROD grid cells, see Section 1.3.1. Note that GPCP grid cells at lower latitudes contain more NIMROD grid cells and that the threshold of 190 is about one third of the maximum number of NIMROD grid cells in one GPCP grid cell, whereas the highest values in the actually covered area is less than 500 NIMROD grid cells, see panel A.

Supplementary Figure 7: Same as Supplementary Figure 6, but for NIMROD grid cells contained in the 2.5° GPCP grid cells for the monthly averaged product (i.e., based on the data availability outlined in Supplementary Figures 4 and 5). Here, the respective threshold below which data are discarded is 1140 NIMROD grid cells, see Section 1.3.1. Note that GPCP grid cells at lower latitudes contain more NIMROD grid cells and that the threshold of 1140 is about one third of the maximum number of NIMROD grid cells in one GPCP grid cell, whereas the highest values in the actually covered area is less than 3000 NIMROD grid cells, see panel A.

References

Adler, R.F.; Huffman, G.J.; Chang, A. et al., 2003: The Version-2 Global Precipitation Climatology Project (GPCP) Monthly Precipitation Analysis (1979–Present). J. Hydrometeor., 4, 1147–1167, https://doi.org/10.1175/1525-7541(2003)004<1147:TVGPCP>2.0.CO;2.

Adler, R.F.; Gu, G.; Sapiano, M. et al., 2017: Global Precipitation: Means, Variations and Trends During the Satellite Era (1979–2014). Surv Geophys 38: 679. https://doi.org/10.1007/s10712-017-9416-4.

Adler, R.F.; Sapiano, M.R.P.; Huffman, G.J. et al., 2018: The Global Precipitation Climatology Project (GPCP) Monthly Analysis (New Version 2.3) and a Review of 2017 Global Precipitation. Atmosphere, 9, 138, https://doi.org/10.3390/atmos9040138.

Behrangi, A.; Gardner, A.; Reager, J.T. et al., 2018: Using GRACE to Estitmate Snowfall Accumulation and Assess Gauge Undercatch Corrections in High Latitudes. J. Climate, 31, 8689–8704, https://doi.org/10.1175/JCLI-D-18-0163.1.

Bolvin, D.T.; Adler, R.F.; Huffman, G.J. et al., 2009: Comparison of GPCP Monthly and Daily Precipitation Estimates with High-Latitude Gauge Observations. J. Appl. Meteor. Climatol., 48, 1843–1857, https://doi.org/10.1175/2009JAMC2147.1.

Bosilovich, M.G.; Chen, J.; Robertson, F.R.; Adler, R.F., 2008: Evaluation of Global Precipitation in Reanalyses. J. Appl. Meteor. Climatol., 47, 2279–2299, https://doi.org/10.1175/2008JAMC1921.1.

Burdanowitz, J.; Klepp, C.; Bakan, S.; Buehler, S.A., 2018: Towards an along‐track validation of HOAPS precipitation using OceanRAIN optical disdrometer data over the Atlantic Ocean. Q. J. R. Meteorol. Soc., 144:235–254. https://doi.org/10.1002/qj.3248.

Copernicus Climate Change Service (C3S), 2017: ERA5: Fifth generation of ECMWF atmospheric reanalyses of the global climate. Copernicus Climate Change Service Climate Data Store (CDS), 21/12/2018. https://cds.climate.copernicus.eu/cdsapp#!/home.

Dinku, T.; Ceccato, P.; Grover‐Kopec, E. et al., 2007: Validation of satellite rainfall products over East Africa's complex topography, Int. J. Remote Sens., 28:7, 1503-1526, https://doi.org/10.1080/01431160600954688.

GCOS (2016), THE GLOBAL OBSERVING SYSTEM FOR CLIMATE: IMPLEMENTATION NEEDS, GCOS GCOS-200, GOOS-214, https://library.wmo.int/doc_num.php?explnum_id=3417

Goddard Earth Sciences Data and Information Services Center, 2016: TRMM (TMPA) Precipitation L3 1 day 0.25 degree x 0.25 degree V7, Edited by Andrey Savtchenko, , Goddard Earth Sciences Data and Information Services Center (GES DISC), https://doi.org/10.5067/TRMM/TMPA/DAY/7.

Greene, J.S.; Klatt, M; Morrissey, M.; Postawko, S., 2008: The Comprehensive Pacific Rainfall Database. J. Atmos. Oceanic Technol., 25(1), 71-82, https://doi.org/10.1175/2007JTECHA904.1.

Huffman, G.J.; Adler, R.F.; Morrissey, M.M. et al., 2001: Global Precipitation at One-Degree Daily Resolution from Multisatellite Observations. J. Hydrometeor., 2, 36–50, https://doi.org/10.1175/1525-7541(2001)002<0036:GPAODD>2.0.CO;2.

Immler, F.J.; Dykema, J.; Gardiner, T. et al, 2010: Reference Quality Upper-Air Measurements: guidance for developing GRUAN data products, Atmos. Meas. Tech., 3, 1217-1231, https://doi.org/10.5194/amt-3-1217-2010.

Joshi, M.K.; Rai, A.; Pandey, A.C., 2013: Validation of TMPA and GPCP 1DD against the ground truth rain‐gauge data for Indian region. Int. J. Climatol, 33: 2633-2648. https://doi.org/10.1002/joc.3612.

Klepp, C., 2015: The oceanic shipboard precipitation measurement network for surface validation – OceanRAIN. Atmospheric Res., 163, 74-90, https://doi.org/10.1016/j.atmosres.2014.12.014.

Klepp, C.; Michel, S.; Protat, A. et al., 2017: Ocean Rainfall And Ice-phase precipitation measurement Network - OceanRAIN-W. World Data Center for Climate (WDCC) at DKRZ. https://doi.org/10.1594/WDCC/OceanRAIN-W.

Klepp, C.; Michel, S.; Protat, A. et al., 2018: OceanRAIN, a new in-situ shipboard global ocean surface-reference dataset of all water cycle components. Scientific Data, 5, 180122, https://doi.org/10.1038/sdata.2018.122.

Masunaga, H.; Schröder, M.; Furuzawa, F. et al., 2019: Inter-product biases in global precipitation extremes. Environ. Res. Lett., 14, 125016, https://doi.org/10.1088/1748-9326/ab5da9.

McPhee, J.; Margulis, S.A., 2005: Validation and Error Characterization of the GPCP-1DD Precipitation Product over the Contiguous United States. J. Hydrometeor., 6, 441–459, https://doi.org/10.1175/JHM429.1.

Met Office (2003). Met Office Rain Radar Data from the NIMROD System. NCAS British Atmospheric Data Centre, http://catalogue.ceda.ac.uk/uuid/82adec1f896af6169112d09cc1174499.

Pfeifroth, U.; Mueller, R.; Ahrens, B., 2013: Evaluation of Satellite-Based and Reanalysis Precipitation Data in the Tropical Pacific. J. Appl. Meteor. Climatol., 52, 634–644, https://doi.org/10.1175/JAMC-D-12-049.1.

Sun, Q.; Miao, C.; Duan, Q. et al., 2018: A review of global precipitation data sets: Data sources, estimation, and intercomparisons. Rev. Geophys., 56, 79-107. https://doi.org/10.1002/2017RG000574.

Szczypta, C.; Calvet, J.-C.; Albergel, C. et al., 2011: Verification of the new ECMWF ERA-Interim reanalysis over France, Hydrol. Earth Syst. Sci., 15, 647-666, https://doi.org/10.5194/hess-15-647-2011.

Tropical Rainfall Measuring Mission (TRMM), 2011: TRMM (TMPA/3B43) Rainfall Estimate L3 1 month 0.25 degree x 0.25 degree V7, Greenbelt, MD, Goddard Earth Sciences Data and Information Services Center (GES DISC), https://doi.org/10.5067/TRMM/TMPA/MONTH/7.

Wang, C.; Tang, G.; Han, Z. et al., 2018: Global intercomparison and regional evaluation of GPM IMERG Version-03, Version-04 and its latest Version-05 precipitation products: Similarity, difference and improvements, J. Hydrol., 564, 342-356, https://doi.org/10.1016/j.jhydrol.2018.06.064.

You, Q.; Min, J.; Zhang, W. et al., 2015: Comparison of multiple datasets with gridded precipitation observations over the Tibetan Plateau. Clim. Dyn., 45:791. https://doi.org/10.1007/s00382-014-2310-6.

Related articles

• Page:
  ECV - Glaciers, Ice Sheets, Lakes, Soil Moisture
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product User Guide Specification (PUGS)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: System Quality Assurance Document (SQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assurance Document (PQAD)
• Page:
  IV Version 1.2; SEC Version 2.0; GMB Version 2.0: Product Quality Assessment Report (PQAR)