...

History of modifications

List of datasets covered by this document

Anchor
related_documents related_documents

Related documents

Acronyms

Scope of the document

This document is the Algorithm Theoretical Basis Document (ATBD) for the Copernicus micrOwave-based gloBal pRecipitAtion (COBRA) product. Provided are gridded (level 3) daily and monthly estimates of precipitation rates, derived by combining precipitation information from two sources:

...

The HOAPS algorithm is described in its dedicated ATBD [D1] and only summarized here.

The document includes the description of gridding, post-processing and merging procedures for combining observations obtained through HOAPS and PNPR-CLIM into the final COBRA product.

Executive summary

Precipitation rate estimates obtained through HOAPS and PNPR-CLIM are combined to form the Copernicus micrOwave-based gloBal pRecipitAtion (COBRA) dataset.

...

Monthly gridded estimates of precipitation are obtained by first averaging over all instantaneous precipitation rate estimates stored in the hourly gridded files for each available platform. Again, these are weighted by the overlap area between the corresponding observational footprint and the grid cell. Finally, the monthly gridded precipitation estimates are averaged over all available platforms by weighting the monthly contribution from each platform by the availability of the respective platform in each month. As in the daily data set, the same indicators of spatiotemporal variability and quality are provided.

Anchor
section1 section1

1 Instruments

Anchor
section1_1 section1_1

1.1 Instruments used within the PNPR-CLIM algorithm

The PNPR-CLIM algorithm is based on the AMSU-B (on board NOAA-15, NOAA-16, NOAA-17 satellites) and MHS (on board NOAA-18, NOAA-19, and MetOp satellites) microwave sounders. The local equator crossing time of the descending/ascending node for all NOAA satellites lies between 5:00–9:15 a.m./p.m., while for the MetOp satellites it lies between 8:45–9:30 a.m/p.m.. These cross-track scanning radiometers provide measurements at 90 steps of constant 1.1° angular sampling across track, which implies that the IFOV elongates as the beam moves from nadir toward the edge of the scan.

...

The sampling distance also varies with the scan angle and corresponds to the sampling geometry of AMSU-B/MHS (1.1 degrees), which corresponds to 16 km at nadir. Table 1 presents some AMSU-B and MHS radiometer characteristics.

...

The detailed description of them and their role in the algorithm development can be found in section 3.2.

Anchor
section1_2 section1_2

1.2 Instruments used in the HOAPS-v4 algorithm

Anchor
section1_2_1 section1_2_1

1.2.1 The SSM/I Instrument

The SSM/I is a microwave radiometer, which measures micro wave emission and scattering at 7 channels with four centre frequencies, 19.35, 22.235, 37.0 and 85.5 GHz. A summary of frequency and channel characteristics is shown in table 2.

The SSM/I instrument has been used within the DMSP aboard F-8, F-10, F-11, F-13, F-14 and F-15 spacecrafts. Apart from F-8, local equator crossing time of the descending/ascending node lies for all satellites between 5–10 a.m./p.m. In case of F-8, descending and ascending node were reversed. Thus, local equator crossing time was about 6 a.m. for the ascending node of F-8.

The satellites fly on a near-polar, sun-synchronous, circular orbit, whose period is about 102 minutes. This results in about 14.1 orbits per day and uncovered regions poleward of 87.5°.

Table 3 gives a short summary of further instrument characteristics. A full description of the SSM/I instrument is given by Hollinger (1987, 1990) and Wentz (1991).

...

Anchor
section1_2_2 section1_2_2

1.2.2 The SSMIS Instrument

The SSMIS is a 24-channel microwave radiometer using multiple frequencies and therefore is able to replace different former instruments namely SSM/I, Special Sensor Microwave Temperature (SSM/T), and Special Sensor Microwave Humidity Sounder (SSM/T-2). For precipitation retrieval the channels 12–18, with centre frequencies at 19.35, 22.235, 37.0 and 91.655 GHz, are of main interest here. Thus, SSMIS continues the measurements of the SSM/I instrument with a shift of SSM/I highest frequency channel from 85.5 to 91.655 GHz. Further channel characteristics are given in table 4.

Its first usage was aboard the F-16 spacecraft of the DMSP in October 2003. Further missions followed by F-17 (2006), F-18 (2009) and F-19 (2014). Like earlier satellites of the DMSP (see section 1.2.1) these ones are on a near-polar, sun-synchronous, circular orbits with a period of 101.8 minutes. Their local equator crossing occurred during 2–11 a.m./p.m.

Table 5 summarizes further instrument characteristics. F-19 was out of control since 2016, thus its data might be of lower quality [D1].

A full description of the SSMIS instrument is given by Northrop Grumman Corporation (2002) and Kunkee (2008).

...

Anchor
section1_2_3 section1_2_3

1.2.3 The AMSR-E Instrument

The AMSR-E is a passive microwave radiometer with twelve channels at six frequencies. Its centre frequencies are located at 6.925, 10.65, 18.7, 23.8, 36.5, and 89.0 GHz. Each frequency band operates in dual-polarization, i.e., channels of horizontal and vertical polarization are measured separately. AMSR-E is a conical scanning instrument. Sensor characteristics are summarised in table 6.

The instrument is mounted on the Aqua spacecraft, which was launched in May 2002. Aqua's orbits are near-polar and sun-synchronous with an eccentricity of 0.0015. Equator overpasses of Aqua appear near 1.30 a.m./p.m. local time for the ascending/descending node. Table 7 presents additional information about the Aqua spacecraft and the AMSR-E instrument.

...

The IFOVs of AMSR-E and SSM/I / SSMIS were harmonized by averaging the brightness temperatures of each three neighbouring AMSR-E scan positions.

Anchor
section1_2_4 section1_2_4

1.2.4 The TMI Instrument

TMI is a passive microwave radiometer based on the characteristics of the earlier instrument SSM/I (see section 1.2.1). Compared to SSM/I, the TMI frequencies are expanded with an additional channel at 10.65 GHz. Moreover, the SSM/I channel at 22.235 GHz has been replaced with a new channel centred at 21.3 GHz. Table 8 summarizes more information on TMI channels.

The instrument was onboard the TRMM satellite, which operated from 1997 until April 2015. TRMM had a circular, non-sun-synchronous orbit with an inclination of 35 degrees to the Equator. Additional details about the spacecraft and instrument are shown in table 9 (Kummerow et al., 1998).

...

Anchor
section2 section2

2 Input and Auxiliary Data

Anchor
section2_1 section2_1

2.1 PNPR-CLIM

Anchor
section2_1_1 section2_1_1

2.1.1 Microwave Brightness Temperatures FCDR

Carefully calibrated and homogenised radiance datasets are a fundamental prerequisite for climate studies, climate monitoring and reanalysis. Climate research requires long-period, consistent and uncertainty-quantified data records that the available operational datasets do not provide for several reasons. There are biases between instruments and biases in measurements from the same instrument for different time periods/regions, due to temporary instrument failures. Moreover, the measurement noise may vary during instrument lifetime and calibration procedures also contribute to the overall uncertainty (Hans et al., 2017, 2018, 2019; Brogniez et al., 2016; Merchant et al., 2017; Burgdorf et al., 2018).

The FIDelity and Uncertainty in Climate data records from Earth Observations (FIDUCEO¹) project, created to address these issues, delivered various climate datasets from Earth Observation Satellites, which have received rigorous harmonization treatments between various datasets with a specific analysis of the relative uncertainties (Merchant et al., 2019). The datasets include FCDRs containing harmonised radiances and Climate Data Records (CDRs).

...

Anchor
section2_1_2 section2_1_2

2.1.2 ECMWF ERA5 auxiliary input variables

Table 10 presents some model-derived variables used by the algorithm in addition to the input BTs. The selected variables were obtained from the ECMWF ERA5 monthly and daily mean product at 0.25° × 0.25° resolution.

...

Anchor
section2_2 section2_2

2.2 HOAPS-v4

The HOAPS v4 Level-2 data have been generated on the basis of the CM SAF SSM/I / SSMIS FCDR (Fennig et al., 2017; Fennig et al., 2020; CM SAF’s HOAPS v4.0 ATBD [D1], section 2.2). The initial dataset was extended to cover the full period until the end of 2017 for the present merged CDR. It is assumed that the inter-calibration coefficients remain applicable over the period 2015–2017.

...

HOAPS products are only available over ice-free ocean. While the sea ice is identified in the observed brightness temperatures internally in the algorithm, the global land masses are filtered out using an auxiliary land/ocean mask.

Anchor
section2_3 section2_3

2.3 Gridding, post-processing and merging

The Level 2 files, which are the output of the PNPR-CLIM and HOAPS algorithms (see Appendix), form the input to the gridding, post-processing, and merging procedures.

Anchor
section3 section3

3 Algorithms

Anchor
section3_1 section3_1

3.1 The processing chain

Figure 1 provides an overview of the processing chain with all relevant algorithms and respective in- and output data for the generation of COBRA. First, instantaneous precipitation rate estimates are derived from brightness temperatures and respective auxiliary data through the PNPR-CLIM (section 3.2) and HOAPS (section 3.3) algorithms (depending on the data source). These are then gridded to a 1° x 1° hourly global grid (FPG algorithm, section 3.4.1) and post-processed (bias correction and filtering for low-quality periods for certain platforms, section 3.4.2). In a last step, the hourly gridded intermediate results from various platforms are merged and agglomerated to daily (section 3.4.3) and monthly (section 3.4.4) fields.

Figure 1: Schematic of the processing chain for the generation of COBRA data products. The central panels with dark grey background indicate the input/output data at various levels. The panels to the sides with light grey background indicate the actual processing steps. The colors of processing arrows and respective algorithm box match. The sections in this document, where the respective algorithms or data are explained in detail, are named where applicable.

Anchor
section3_2 section3_2

3.2 PNPR-CLIM

Anchor
section3_2_1 section3_2_1

3.2.1 Theoretical Basis

Artificial neural networks (NNs) represent a highly flexible ensemble of non-linear and non-parametric regression and classification statistical models, increasingly applied in environmental sciences for their capability to approximate complex non-linear and imperfectly known functions to an arbitrary degree of accuracy (e.g., Liou et al., 1999; Aires et al., 2001; Blackwell and Chen, 2005). The opportunities offered by their ability to learn and generalize, as well as their robustness to noise, have encouraged their use in precipitation estimation retrieval from satellite and ground-based measurements. NN techniques have proven to be effective in this research area and have been successfully used in many rainfall estimation and monitoring applications (e.g., Hong et al., 2004; Surussavadee and Staelin, 2008; Mahesh et al., 2011; Tapiador et al., 2017).

...

A NN requires a large sample of observational data, wide enough to be representative of the true population, comprehensive of both predictors (e.g., BTs) and predictands (e.g., precipitation rates). During the training phase the network learns the intrinsic correlations among the observed and the hidden variables, by adjusting its inner parameters to increase the prediction accuracy. It consists of a sequence of layers connected through compositions of (parametric) affine transformations with certain (fixed) non-linear transfer functions, mapping a multi-dimensional vector space into another, whose components are called neurons (also perceptrons). An illustrative scheme of a NN is shown in figure 2, with the input layer receiving the input signals, the hidden layer(s) and the output layer, providing the network response.

...

where M is the number of elements of the training set. The network corrects its weights to lessen the errors through an iterative process aimed at the minimization of the error. At the end of the training, the final values of the weights connecting the neurons of the different layers, store the knowledge of the NN (McCann, 1992). The design of the network architecture is normally quite complex. The model selection in NN aims at finding as few hidden units and neuron-neuron connections as necessary for a good approximation of the true function.

Anchor
section3_2_2 section3_2_2

3.2.2 NN Training

The approach based on NNs requires a "training phase", that uses a large sample of data representative of the input and output variables of the retrieval process (in this case the BTs with ancillary parameters and the surface precipitation rate, respectively). The performance of the NN is largely dependent on the completeness and representativeness of the database and on its consistency with the actual observations.

...

Since the launch of the Global Precipitation Measurement mission (GPM) on February 28, 2014, quasi-global high quality spaceborne radar precipitation measurements have become available. The Dual-frequency Precipitation Radar (DPR) onboard the GPM Core Observatory (GPM-CO) covers the area between 67 °N and 67 °S of the globe. The high quality of the precipitation measurements is supported by several validation (Schwaller et al., 2011, Kim et al., 2014, Speirs et al., 2017) and field campaigns (Lee et al., 2019; Houze et al., 2017; Tao et al., 2016). For the development of the PNPR-CLIM algorithm an observational dataset, built from coincident, in space and time, DPR precipitation measurements with the MHS radiometer measurements (BTs), has been used in the NN design phase.

Anchor
section3_2_2_1 section3_2_2_1

3.2.2.1 The Dual-frequency Precipitation Radar

The GPM-CO DPR is the second space-borne precipitation radar, following the Precipitation Radar launched on the TRMM satellite in November, 1997. The DPR consists of a Ku-band (13.6 GHz) and a Ka-band (35.5 GHz) radars. These Earth-pointing KuPR and KaPR instruments provide 3D precipitation measurements over all surfaces between 67 °N and 67 °S since March 2014. The KuPR and KaPR design specifications are shown in table 11.

Anchor
section3_2_2_2 section3_2_2_2

3.2.2.2 The NN training dataset

Table 12 presents the main characteristics of the MHS-DPR coincidence database used for the NN design. This dataset was built as follows. Coincidences between NOAA-18, NOAA-19, MetOp-A, MetOp-B MHS measurements and DPR Ku-band measurements within a time interval of 15 minutes were considered for the creation of the database. The database covers the period from 1 January 2015 through 31 December 2016 (24 months). The GPM level-2 precipitation product obtained by combining the GMI and DPR measurements (Grecu et al., 2016) (2B-CMB, version 06A) is used as reference. In particular, the precipitation estimates used in the observational database are provided on the Ku-band radar swath (245 km wide) and obtained from the DPR Ku-band reflectivity and GMI brightness temperatures. The observational database is made of co-located vectors of MHS BTs (from the FIDUCEO dataset, see section 2.1.1) and 2B-CMB surface precipitation rate spatially averaged to match the MHS IFOV (variable along the scan line). Some model-derived variables (from ECMWF ERA5, see section 2.1.2) have been added to the database (see table 13) to be used, together with the input BTs, in the algorithm.

...

Anchor
section3_2_3 section3_2_3 _heading h.17dp8vu

3.2.3 Algorithm Flowchart

The PNPR-CLIM algorithm high-level flowchart is shown in figure 3.

...

In the present scheme the algorithm takes as input the FIDUCEO v4.1 BTs of the MHS and AMSU-B radiometers by checking the quality of the input data (quality control module), some ancillary information regarding the thermodynamic state of the atmosphere (from the ECMWF ERA5 model), and regarding the state of the background surface. All the inputs feed the precipitation classification module that is optimized for the detection of the precipitation and its classification. The BTs arrays and the corresponding ancillary data of pixels classified as precipitating, feed the precipitation rate estimate and calibration module. The quality index module evaluates a pixel-based quality flag using the quality of the input data and the accuracy of the retrieval in different meteorological and environmental conditions (e.g. presence of ice/snow, dry condition, and strong convection).

Anchor
section3_2_4 section3_2_4 _heading h.26in1rg

3.2.4 Precipitation classification module

Anchor
section3_2_4_1 section3_2_4_1 _heading h.tb9vdh8j2zze

3.2.4.1 Introduction

In general, the identification of precipitation areas, or Precipitation/No-precipitation Classification (PNC) of pixels, represents a preliminary step to the MW precipitation retrieval and is considered crucial to obtain good performances in passive microwave precipitation retrieval (Ferraro et al., 1998; Seto et al., 2008; Sudradjat et al., 2011; Kirstetter et al., 2013; Kacimi et al., 2013). Therefore, the success of any MW retrieval algorithm relies on proper identification of precipitating pixels and the screening of non-precipitating pixels that might produce a signature similar to that of precipitation (Ferraro et al., 1998). For example, over land, the PNC discrimination is difficult due to the high variability of ground emissivity (Grecu and Anagnostou, 2001). This filtering process is therefore critical for instantaneous retrievals but even more so when developing accumulated rain products (Kacimi et al., 2013). PNC, in general, assigns a deterministic flag for precipitation or no-precipitation to each pixel; then, only observations with a rain flag are processed in the precipitation retrieval module.

Anchor
section3_2_4_2 section3_2_4_2 _heading h.549sa31ve5a5

3.2.4.2 PNC Module structure

PNC module consists of a stand-alone NN classifier with 2 hidden layers of 45 and 15 units, and sigmoid transfer functions. Its output turns out to be a continuous function with values in the range [0, 1] which, under suitable hypotheses on the training dataset distribution (see Bishop, 1995, for more details), approximates the probability of precipitation given the input observation. With this interpretation in mind, the threshold value 0.5 is used to distinguish precipitating (> 0.5) and non-precipitating states (≤ 0.5).

The network ingests three types of input variables: instantaneous, average, and static variables (see table 14). The instantaneous variables are the AMSU-B / MHS FCDR BTs (at 89, 150/157, 1831, 1833 and 190/1837 GHz). The monthly variables from the ERA5 reanalysis include the 2 m Temperature (T2m, K), Freezing Level (FL, m), Total Precipitable Water Vapor (TPWV, kg m^-2), Snow Depth (SD, cm) and Sea-Ice fraction (SIF, dimensionless). Finally, the static variables are the (secant of the) scan angle (ANG) and the surface type (Ocean, Land or Coast (OLC)).

...

The network architecture described above was not arbitrarily chosen: several different combinations of layers, units per layer and activation functions were tested using the 2015 DPR-MHS coincidence dataset (training dataset, see section 3.2.2.2).

Anchor
section3_2_4_3 section3_2_4_3 _heading h.m52xjfm2j82x

3.2.4.3 PNC Module performance verification

The performance analysis of PNC module was carried out using the 2016 DPR-MHS coincidence dataset to verify consistency and stability of the NN performance.

Before proceeding with the module assessment, let us introduce some further notation. Usually, for two-classes classification problems, where the target variable t and the prediction variable y assume binary values (1 for rain and 0 for no rain), the validation dataset (VD) can be divided into four disjoint subsets forming the contingency table defined by eqs. 3.4:

...

Considering the statistical parameters defined in eqs. 3.4, several indices can be computed. The accuracy (acc), probability of random agreement (rnd) and the Cohen's kappa (𝜅, Cohen, 1960) are defined as follows (where the modulus denotes the size):

...

For the PNC classifier, we expected that, due to resolution and channel assortment limitation (specifically the lack of low frequency channels, extremely useful for robust precipitation retrievals over ocean), very light precipitation detected by the DPR could be easily missed by the MHS and thus by any algorithm based on its observations. To highlight this behaviour and identify a sensitivity threshold, the Cohen's kappa was evaluated at various minimum precipitation rates (identifying the targets t = 1). Please, note that varying the minimum precipitation rate does not change the proportion of the predicted positives/negatives. Therefore, in order to balance the effect of introducing fictitious false alarms by increasing the detection threshold (small rates correctly identified as non-zero), the various indices were computed for 2B-CMB rate either equal to 0 mm/h or greater than the chosen minimum threshold. The results are shown in figure 4.

Figure 4: Sensitivity study for the PNC module through the analysis of the Cohen's kappa (blue dots) at various rainfall threshold values. The peak

...

On the validation dataset the PNC module had a far of about 0.20. The pod can be evaluated for different thresholds. In this case, the index represents the probability of detecting rainy samples within a prescribed range, i.e., precipitation rates greater than a minimum threshold value, as shown in figure 5. It can be seen that, at the sensitivity threshold of 0.30 mm/h, the pod turned out to be 0.79. For higher rates, greater than or equal to 0.50 mm/h, its value was 0.87.

...

and acc = 0.79. False alarm rate, on the other hand, was about 0.20. At the same time, pod values above 0.80 were found for precipitation rates greater than or equal to 0.34 mm/h, increasing to values above 0.87 for precipitation rates greater than 0.50 mm/h.

Anchor
section3_2_5 section3_2_5 _heading h.lnxbz9

3.2.5 Precipitation rate estimation module

The design of the NN the for the precipitation rate estimation (PRE) module of PNPR-CLIM algorithm has exploited the experience gained at the CNR-ISAC in the development of NN-based precipitation retrieval algorithms for cross-track scanning MW radiometers (Sanò et al., 2015, 2016). These algorithms (the PNPR v1 for AMSU/MHS and PNPR v2 for ATMS) were developed in the frame of the EUMETSAT H SAF to deliver H SAF L2 operational products P-IN-MHS (H02B) and P-IN-ATMS (H18) over the MSG full disc area (Mugnai et al., 2013). It is important to stress that while these algorithms use a model-based training dataset optimized for European and African regions for near real time applications, the PNPR-CLIM approach is based on the use of a global GPM-based observational dataset and on climatological ancillary data, for climatological applications.

...

The PRE module optimal NN consists of a stand-alone NN model with 2 hidden layers of 28 and 8 units, and sigmoid transfer functions. The NN has been developed using the 15 input variables shown in table 15.

...

The performance analysis of the PRE module was carried out using the 2016 dataset, which is an independent part of the observational MHS-DPR coincidence database, not used in the training and design phase of the algorithm (about 1.5 million points) (see section 3.2.2.2).

Figure 6: Number of occurrences (pixels) vs bins (1 mm/h) of precipitation values retrieved by the NN (PRE module) (red) and those from 2B-CMB in the verification database (blue). The left panel refers to ocean and the right one to land.

Figure 6 shows, in a bar graph, the comparison between the number of occurrences of the precipitation values provided by the NN (red) and those in 2B-CMB (blue), as a function of the precipitation in bins of 1 mm/h for ocean and land. A good agreement is seen between the NN-derived values and the 2B-CMB verification database across all bins, especially over oceans. Some small differences (mainly for land) for high precipitation values (> 15 mm/h) are essentially due to the low number of occurrences (less than 10³). For small values of precipitation (0.1 to 5.0 mm/h), where the number of occurrences is larger (10⁴ – 10⁶), the agreement is very good.

...

Another result of the verification study is shown in figure 7. It shows the 2D histogram of the surface precipitation rate estimates from the NN and the corresponding values in the 2B-CMB dataset over ocean and land. Only pixels for which both the neural network and the 2B-CMB provided rainfall estimates ≥ 0.1 mm/h (TP pixels) were considered. In the scatterplot, the logarithmic axes represent the precipitation rate (NN vs. GPM 2B-CMB referred to as DPR), while the colour represents the number of points in the dataset for each 2D precipitation rate bin. Most of the points are close to the main diagonal for both ocean and land, with slight overestimation of very low precipitation (precipitation rate < 0.5 mm/h) over land by the NN.

The values of the statistical indices (hit bias, correlation coefficient (CC), and RMSE) calculated over the entire verification dataset are also provided in table 16, and they confirm the good agreement between the NN retrievals and the verification dataset with very similar performances for both ocean and land pixels.

Figure 8: As in figure 7, with normalized density scatterplots of the NN (PRE module) and 2B-CMB (DPR) mean precipitation rates (ocean on the left, land on the right).

Figure 8 shows the normalized density scatterplot of the NN retrieval of rainfall rates and the corresponding values in the 2B-CMB dataset, for ocean and land surfaces. Normalisation was performed on the number of the 2B-CMB dataset instances in each precipitation rate bin (i.e. by normalising the scatterplots in figure 7 by the sum of instances in each column). In this way, the scatterplot highlights the rain rate distribution regardless of the number of occurrences for the various values of precipitation. This figure also shows the good agreement between the precipitation values resulting from the NN and from the 2B-CMB, with a slight underestimation by the NN for values greater than 1 mm/h.

The results presented, in addition to showing the agreement between the PRE module and 2B-CMB estimates, also evidence the good outcome of the network training, allowing the use of one unique NN over different land-surface types. The NN, applied to the independent verification global dataset with precipitation rates extending over a wide range of values, shows a good ability to retrieve global precipitation without anomalous inhomogeneity in the estimates.

Anchor
section3_2_6 section3_2_6

3.2.6 The deep convection calibration procedure

A procedure to calibrate for deep convection has been developed in order to adjust the PRE module estimates in certain weather conditions (i.e., the presence of deep convection). The NN of the PRE module has been optimized in order to reproduce, in the most reliable way, the rainfall within the training database. However, since different precipitation regimes are not equally well represented in the dataset, the NN tends to optimize the precipitation estimate corresponding to the most frequent conditions. From figure 6 it is evident that the data points corresponding to precipitation rate greater than 10 mm/h are only a small part of the training dataset. This feature can be a weakness during the NN learning phase as far as intense precipitation regimes are concerned (e.g., the presence of deep convection). This issue leads to an underestimation by PNPR-CLIM (with respect to global precipitation datasets) over land areas characterized by deep convection, as shown in Panegrossi et al. (2020, EGU).

...

where f₀  is the identity map and 𝑡 = 𝑡(𝛥𝑇₁₇, 𝛥𝑇₁₃, 𝛥𝑇₃₇, 𝜙) is a convex, smooth function of the BT differences and the latitude 𝜙 such that t = 1 on the region characterized by deep convection (equation (3.10)) and mid-low latitudes (|𝜙| < 60°), and t = 0 outside of those regions. The shape of f_t for different values of its variables is shown in figure 9.

Figure 10 shows the effect of the calibration module in solving the aforementioned overestimation and underestimation problems.

...

Figure 10: Number of occurrences of precipitation rate values retrieved by the PNPR-CLIM (blue) and MRMS (orange). In the left panel PNPR-CLIM without calibration is shown while the right panel shows the calibrated PNPR-CLIM.

Anchor
section3_2_7 section3_2_7 _heading h.44sinio

3.2.7 The quality index

The algorithm provides a quality index to be associated with the estimated value of surface precipitation rate. The quality flag summarizes the product quality and reliability and provides a simple and immediate criterion for the evaluation of the products towards a correct selection and application of the precipitation estimates with respect to the analysed scenario. This index has been constructed based on seven different criteria:

1. Quality of input data: The information provided by FIDUCEO is used to identify the BTs with less reliability (FIDUCEO quality index: 0 = Reliable, 1 = Use with caution, 2 = Unreliable);
2. Background surface index: The quality is reduced for snow-covered background or presence of sea ice. Daily maps of snow/sea ice cover from ECMWF ERA5 are used to identify these conditions;
3. Orography index: The quality index is reduced if the standard deviation of the terrain elevation within the pixel exceeds a certain threshold (400 m). The ETOPO1 land topography was exploited (Amante et al., 2009);
4. Radiometer Scan index: The quality index is reduced for observations taken at the 5 outermost pixels of the scanline (at the lowest spatial resolution);
5. Precipitation Probability Index: The quality index is reduced if the probability of precipitation is between 30 % and 70 %. In this range, the rain/ no rain discrimination has greater uncertainty;
6. Calibration Index: The quality index is reduced if the BTs belong to the deep convection region identified by equation (3.10);
7. High Latitudes Index: The quality index is reduced at high latitudes(ϕ>60°) in extremely cold and dry conditions. In these conditions the background surface contaminates the precipitation signal, leading to an overestimation of the precipitation and/or false detection. This effect is identified by using a threshold on the 89 GHz channel (BT_89GHz < 175 K).

These criteria make it possible to create two quality indices, The Quality Flag Index (QF) and the Bit Quality Flag Index (BQF). BQF is a bit flag: the positions of the bits with value 1 indicate which of the seven conditions listed above have occurred (see table 17). QF is an integer variable, built from BQF, with values ranging from 0 (high quality) to 3 (poor quality); a further value (4) indicates invalid data. It is defined as the number of non-zero bits of BQF (with maximum allowable value QF = 3). It is also worth pointing out that, if the third (snow-covered surface) or fourth (sea ice) bit of BQF is non-zero, then QF is set to 3 by default because the snow/ice background significantly decreases the capability of predicting precipitation rates. Figure 11 shows the distribution of the QF index for the year 2017.

...

Figure 11: Distribution of the QF index (percentage over all the available observations) for the year 2017.

Anchor
section3_3 section3_3

3.3 HOAPS v4

For HOAPS-v4, a NN algorithm is used to quantify precipitation from the SSMI(S) FCDR BTs. The neural network was trained with precipitation rates retrieved from assimilated brightness temperatures in a 1D-Var scheme from ECMWF. The training data set is based on radiative transfer calculations as described in Bauer et al. (2006a, b). The data set contains one month (August 2004) of assimilated SSM/I brightness temperatures and the corresponding ECMWF 1D-Var retrieved precipitation values of the ECMWF model. For more details on the neural network architecture of the precipitation retrieval algorithm see [D1] and Andersson et al. (2010). Output is provided over ice-free ocean. For precipitation, a sensitivity threshold is implemented at 0.3 mm/h. Below this threshold, a level 2 observation is considered as non-precipitating and consequently set to zero. The algorithm itself was developed outside the C3S project.

Anchor
section3_4 section3_4

3.4 Gridding and merging to produce the COBRA precipitation dataset

Anchor
section3_4_1 section3_4_1

3.4.1 Generation of per-satellite hourly gridded data (FPG algorithm)

Figure 12: Schematic of the FPG algorithm.

...

0.5 \cdot 79.08 + 2.84 \cdot nb - 14.78 \cdot nb^{0.666}

0.5 \cdot 28.72 - 0.90 \cdot nb + 0.094 \cdot nb^{1.5}

...

When generating L3 (i.e. spatially and temporally gridded) daily data, the L2 instantaneous precipitation rates are first averaged in hourly intervals on the final 1° × 1° latitude-longitude grid (referred to as 1 degree hourly (1DH)). This intermediate step is carried out so that diurnal cycles are represented optimally wherever the L2 observational database allows (see sections 3.4.3 and 3.4.4). The monthly data are computed as averages of the instantaneous precipitation rates directly. However, the L2 instantaneous precipitation rates have not been processed twice (for hourly and monthly averaging, respectively). Instead, the monthly averaging uses intermediate results from the hourly binning (see below).

...

The FPG algorithm processes one day of L2 data for one platform. Figure 12 shows a flow chart for FPG. It consists of the following modules:

• L2 Input: FPG reads one-dimensional arrays containing scan line time and scan position and two-dimensional (scan line vs. scan position) arrays for precipitation rates in mm/h, latitudes and longitudes of footprint centres, and – in the case of PNPR-CLIM – quality flag (see section 3.2.7).

• Grid set-up: The 1° × 1° grid is defined in terms of grid cell edges and centre points. For each grid cell, the European Petroleum Survey Group (EPSG) code of the respective optimal Universal Transversal Mercator (UTM) zone is determined, based on the geographical coordinates of the grid cell's centre point. The grid cell's outline is sampled regularly by npe points per edge in latitude-longitude coordinates, which are then transformed into the grid cell's outline polygon in the respective UTM coordinates in m. In the following, we opt for n_pe = 5.

• Footprint set-up: The ellipses of the footprints are set-up in Cartesian coordinates. The semi axes are in along-scan and cross-scan directions. Table 18 contains the formula or values for semi axes lengths. In the case of MHS and AMSU-B (PNPR-CLIM), we use the parameterisation of Bennartz (2000). Note that for the cross-track scanning geometry, the along-track direction corresponds to the cross-scan direction, as does the cross-track direction and along-scan direction. In the case of SSM/I and SSMIS observations (HOAPS v4 L2 observations) are independent of the scan position, due to the conical scanning geometry. The values reported in Table 18 correspond to the footprint of the 37 GHz channel of the instruments, as higher-resolution observations are first averaged to these footprints in the HOAPS processing. The ellipses are sampled as polygons with points spaced by 10° increments in the polar angle of the 2-dimensional Cartesian plane.
• Hourly gridding: The L2 data of the considered day (24 h) are sliced in hourly subsets. For each hour, probably covered grid cells are determined as follows:
  1. The polygon formed by the hull of the entire hourly L2 swath (i.e., of all locations for which an observation has been recorded in this hour) is retrieved in terms of latitude-longitude coordinates. Grid cells are tested whether they fall inside the respective polygon. It is possible that many hourly swaths cover at least one of the poles, or cross the longitudinal ±180° boundary, where such a polygon would not be well defined. In this case the swath and the grid cells are thus rotated towards the equator inside the ±180° longitude limits. Grid cells that fall inside the polygon or have a minimum distance to the edge of the swath of less than 250 km (evaluated using the Haversine formula for the grid cell centre points and the footprint centre points of the outmost scan positions) are considered as probably covered.
  2. It can occur, due to the imperfect identification of the orbit situation, that the rotation (mentioned under point a) fails to move the swath away from the poles or the ±180° longitudinal boundary. In these cases, the additional rotation is discarded and the grid cells are instead identified as possibly covered if they have a minimum distance to the edge of the swath of 1200 km in the same sense as above.
  3. Additionally, in the case of HOAPS v4 observations, grid cells not covering open ocean are filtered out.

...

 where sa_max is the maximum length of the semi axes (see table 18) and R = 6371 km is the Earth's radius.

...

Figure 13: Probability density functions (PDF) for precipitation over ocean in the two datasets, based on all available 1DH data (NOAA15 and NOAA16 are excluded in the case of PNPR-CLIM due to lack of overall stability). The various panels cover different latitudinal bands specified in the panels' titles. The colors represent months. Solid lines correspond to PNPR-CLIM; dashed lines correspond to HOAPS. Zero-precipitation events have been excluded. Note the logarithmic scale of the y-axes and the nonlogarithmic, nonlinear scale of the x-axes.

Anchor
section3_4_2 section3_4_2

3.4.2 Post-processing

Anchor
section3_4_2_1 section3_4_2_1

3.4.2.1 Bias correction of 1DH precipitation rates

The overall distributions of hourly gridded data produced by FPG (section 3.4.1) differ strongly for the PNPR and HOAPS datasets over the ocean, see figure 13. It was decided to harmonise the datasets over the ocean. A dataset can be manipulated such that its distribution matches that of another one by quantile mapping, i.e., based on cumulative probabilities. In our case, this means summing up the PDFs displayed in figure 13. For a given month and latitudinal band, we then have a cumulative PDF for HOAPS, f_H, and one for PNPR, f_P. For a given 1DH non-zero rate of precipitation in HOAPS, p_H, the corresponding precipitation rate according to the PNPR PDF is

...

. Mapping function based on the PDFs in figure 13 are illustrated in figure 14.

Figure 14: Mapping of 1DH precipitation rates over ocean between HOAPS (x-axes) and PNPR (y-axes) for the same categories (latitudinal bands in panels, months as colors) as in figure 13, based on the PDFs shown in figure 13. Note the nonlinear scaling of the axes. For an improved interpretability, we are also including black dashed lines corresponding to scaling with a constant factor (see lower left panel for the annotations of these lines), as well as the identity (1:1) mapping as a solid black line. The mapping functions are constructed as described in the main text. At higher precipitation rates, the populations become sparser, which is why a spline fit towards an identity (1:1) mapping has been carried out from the first occurrence of respective discontinuities to avoid an unphysical, spurious mapping.

For example, rates around 1 mm/h in HOAPS are scaled to higher values when mapped to the PNPR distribution at low latitudes (upper right panel in figure 14), or low rates in PNPR are scaled to higher values when mapped to the HOAPS distribution at high latitudes (left panels in figure 14). A comparison of resulting merged daily and monthly values (see sections 3.4.3 and 3.4.4) yielded a general underestimation in high latitudes and a likely underestimation in low latitudes. Therefore, we opted for a bias correction of PNPR 1DH values towards HOAPS in high latitudes and vice versa in low latitudes. In both cases, this generally increases the overall amount of precipitation.

The exact procedure is as follows: For each month, the individual mappings (PNPR-to-HOAPS and HOAPS-to-PNPR) are linearly interpolated in latitude onto the 1° output grid, assuming that the mappings shown in figure 14 are valid at the central latitudes in the respective bands, i.e. at ±70°, ±40°, ±20°, 0°. At latitudes poleward of ±70°, the respective mapping at ±70° is retained. This latitudinal interpolation is carried out to avoid discontinuities in the climatology at the boundaries of the latitudinal bands. HOAPS non-zero 1DH precipitation rates are corrected according to the above described latitudinally interpolated HOAPS-to-PNPR mapping in latitudes between -20° and +20° and mapped to identity (i.e., retained) at latitudes poleward of ±30°. Between ±20° and ±30°, the above (HOAPS-to-PNPR) mapping and the identity mapping are mixed with linearly increasing or decreasing weights. The correction of non-zero PNPR 1DH precipitation rates is implemented inversely, i.e., apply the latitudinally interpolated PNPR-to-HOAPS mapping poleward of ±30°, map to identity (i.e., retain) between -20° and +20°, and mix linearly in-between. Zero precipitation is always retained as zero precipitation.

Anchor
section3_4_2_2 section3_4_2_2

3.4.2.2 Bias correction of 1DH standard deviation and other variables

The standard deviation in a 1DH grid cell (which is part of the FPG output (section 3.4.1)) cannot be corrected as straightforwardly as the mean values, precip_mean (described in section 3.4.2.1). We mapped the precip_mean ± precip_stdv and derived the bias-corrected standard deviation from the resulting difference. However, the standard deviation can occasionally exceed the mean value, such that precip_mean - precip_stdv < 0, which does not make sense for precipitation and the above quantile mapping. In this case, we determine the scaling factor s = precip_stdv / precip_mean by which we have to scale precip_stdv such that precip_mean - precip_stdv / s = 0. We can then bias-correct the values precip_mean ± precip_stdv / s according to section 3.4.2.1, derive the bias-corrected standard deviation from the respective difference, and re-scale by factor s. In cases where precip_mean - precip_stdv ≥ 0, it is s = 1.
The variables pxa and p2xa (see section 3.4.1) need to be biased-corrected, too, so that monthly values are treated similarly. As for 1DH values, it is precip_mean = pxa / norm (see section 3.4.1), the bias-corrected pxa can be computed directly from the bias-corrected 1DH precip_mean (described in section 3.4.2.1).
With the relation between precip_stdv, precip_mean, norm, and p2xa given in section 3.4.1, the bias-corrected p2xa can be computed based on the bias-corrected precip_stdv and precip_mean.

Anchor
section3_4_2_3 section3_4_2_3

3.4.2.3 Filtering

3.4.2.3.1 Filtering of NOAA15 data

Precipitation rates retrieved by PNPR-CLIM using NOAA15 observations prove to be spurious at all available times, with certain drifts occurring later in the time series. Hans et al. (2017, Appendix A5.3) recommend to not use channel 3 data after year 2000. With NOAA16 being available only from early 2001 and because we rely on AMSU-B/MHS observations over land, we retain NOAA15 observations from 2000/01/01 to 2001/03/31, but set the respective quality flag of NOAA15 observations to three (worst quality).

3.4.2.3.2 Filtering of NOAA16 data

NOAA16 observations are not used from 2010/01/01 onwards, due to a drift in respective precipitation rates, see also Hans et al. (2017).

3.4.2.3.3 Filtering of NOAA19 data

NOAA19 observations on 2017/10/09 appeared unreasonably high. All observations on that day were discarded.

3.4.2.3.4 Filtering of AMSR-E data

Due to the higher native resolution of AMSR-E observations, the applied sea-ice mask for the retrieval of precipitation rates in HOAPS creates erroneous values where the signal over sea ice leaks into the averaged brightness temperatures (section 1.2.3). It manifests in extremely high precipitation, mostly in polar latitudes. These outliers are always based on a low number of instantaneous L2 observations. We filter out these erroneous observations by discarding all data falling below the black dashed line in the two-dimensional histogram of high-precipitation occurrences in figure 15.

Anchor
section3_4_3 section3_4_3

3.4.3 Merging and aggregation to daily precipitation

This section describes the computation of a globally gridded precipitation product at a daily time resolution. Variable names given in this section are the same as those given in table 20 in section 4.1 and section 3.4.

Starting with the hourly 1° × 1° gridded data (1DH) from multiple platforms that result from the gridding algorithm described in section 3.4.1, 1DH global/near-global coverage is achieved by merging 1DH precipitation rates from these platforms into one single hourly composite 

...

. For this purpose, the variables computed in section 3.4.1 are averaged over all available platforms in each grid cell, with equal weights for each platform by default. The num_obs variable is handled in a different way as it is the sum of all platforms computed for every grid cell. The variables pxa (precip), p2xa (precip_stdv) and qxa (quality_flag) refer to monthly computations (associated names used in monthly files are given in the brackets) and are irrelevant for the computation procedure of daily values.

...

The first auxiliary variable is derived from hourly values by summation. The other variable is provided by the detection of missing values before the interpolation procedure and it has been combined with time and platform information needed in the composite creation.

Anchor
section3_4_4 section3_4_4

3.4.4 Merging of monthly averages

The computation of a globally gridded precipitation product with a monthly time resolution is described in this section. As in the previous section, the naming of variables will be equal to the names given in table 21 in section 4.2 and section 3.4.
The computation of monthly means of precipitation rates is based on the hourly gridded data described in section 3.4.1, too. Here, the monthly sum of pxa is computed for each platform i and normalized by monthly sums of hourly norm values as shown in following equation:

...

with h denoting the hour within the respective month. This implies that the monthly values are not merely averages of the daily values as outlined in section 3.4.3.
In a similar way and on basis of variable p2xa, a monthly intra-platform standard deviation is calculated for each platform i:

...

From these values, platform composites are finally computed using weighted means. The weights are derived from the number of 1DH observations that a platform has got within the respective month normalized by the total number of available 1DH observations of all platforms in this month.
In addition to the rain rate data, quality variables are provided in the same way as it has been done for daily data (see section 3.4.3). These variables contain:

1. Number of observations used to derive the monthly precipitation rates
2. Monthly amount of hours covered by observations of at least one platform

Since equation (3.18) applies for PNPR-CLIM quality flags qxa (quality_flag) too, the monthly quality_flag variable has been calculated in the same manner as the monthly precip values.

Anchor
section4 section4

4 Output data

Final results of gridding and merging procedures are global level 3 composite of precipitation rates on a one degree spatial grid. These precipitation rates are accumulated on daily time scales and averaged on monthly ones respectively (details in section 3.4.3, 3.4.4). For an easier identification, these two datasets have been given the name Copernicus Microwave-based Global Precipitation (COBRA). The output is provided as netCDF files. Their standard is netCDF4, i.e., netCDF version 4.0. The files are in compliance with CF-1.8 (http://cfconventions.org/) and ACDD 1.3 (https://wiki.esipfed.org/) conventions. A detailed description of output files with daily data can be found below in subsection 4.1. Files of monthly data are described in subsection 4.2.

Table 19 summarizes the valid versions of produced composites of daily and monthly precipitation rates.

...

Anchor
section4_1 section4_1

4.1 Daily merged COBRA data

Files containing daily data of accumulated rain rates are named after following syntax:

...

The short cut 1D for SpatialResolution denotes the latitude-longitude grid of 1° × 1°. Table 20 lists all variables contained in the specific files and gives a short description of them.

...

Anchor
section4_2 section4_2

4.2 Monthly merged COBRA data

Files containing monthly data of average rain rates are named using the following syntax:

...

The short cut 1D for SpatialResolution denotes the latitude-longitude grid of 1° × 1°. Table 21 lists all variables that are contained in these files with a short description of them.

...

Anchor
annex annex

Annex

Anchor
Intermediate_PNPR-CLIM_output Intermediate_PNPR-CLIM_output

Intermediate PNPR-CLIM output

The PNPR-CLIM output is an instantaneous precipitation rate (level 2) product generated from AMSU B and MHS cross-track scanners on board operational satellites in sun-synchronous orbits. The PNPR-CLIM output is provided in netCDF (V4.0) format and is CF V1.8 convention compliant (http://cfconventions.org/).
The input (FIDUCEO FCDR BTs) and output (PNPR-CLIM outputs) filenames have the following structure:

...

Dataset type: Hierarchical Data Format, version 5 
File: PNPR-CLIM_FIDUCEO_FCDR_L1C_MHS_METOPA_20170101101322_20170101115439_EASY_v4.1_fv2.0.1.nc {
  dimensions:
    scan = 2286;
    pos = 90;
    chan = 5;
  variables:
    double pr(scan=2286, pos=90);
      :units = "mm / h";
      :long_name = "Precipitation Rate";
      :coordinates = "utime lat lon";
      :_FillValue = NaN; // double

    double pp(scan=2286, pos=90);
      :long_name = "Probability of Precipitation";
      :coordinates = "utime lat lon";
      :_FillValue = NaN; // double

    double upr(scan=2286, pos=90);
      :_FillValue = NaN; // double
      :units = "mm / h";
      :long_name = "Unmasked Precipitation Rate";
      :description = "Precipitation rate regardless the probability of precipitation";
      :coordinates = "utime lat lon";

    

    short bqf(scan=2286, pos=90);
      :units = "bit";
      :values = "0 : weak probability, 1 : extreme scan position, 2 : inherited fiduceo bitmask, 3 :
      snow, 4 : ice, 5 : orography, 6 : extreme cold , 7 : calibrated, 8 : invalid data";
      :long_name = "Quality Bit Flag";
      :coordinates = "utime lat lon";
      :_FillValue = -1S; // short
      :_Unsigned = "true";

    float lat(scan=2286, pos=90);
      :_FillValue = NaNf; // float
      :units = "North degree";
      :long_name = "Latitude";

    float lon(scan=2286, pos=90);
      :_FillValue = NaNf; // float
      :units = "East degree";
      :long_name = "Longitude";

    double utime(scan=2286);
      :long_name = "Scan time";
      :units = "Seconds since 1970";
      :_FillValue = NaN; // double

    long scan(scan=2286);
      :long_name = "Scan line";

    long pos(pos=90);
      :long_name = "Scan position";

    long chan(chan=5);
      :long_name = "Channel";

  // global attributes:
  :_NCProperties = "version=2,netcdf=4.6.2,hdf5=1.10.4";
  :Source_File = "FIDUCEO_FCDR_L1C_MHS_METOPA_20170101101322_20170101115439_EASY_v4.1_fv2.0.1.nc";
  :Retrieval_Algorithm = "PNPR";
  :Author = "ISAC-CNR";
  :Date_of_Creation = "Thu Jul  2 23:59:20 2020";

Anchor
Intermediate_HOAPS_output Intermediate_HOAPS_output

Intermediate HOAPS output

The HOAPS algorithm output files are Level 2 files, which contain – among others – precipitation, geographical coordinates of each observations centre point grouped in the two dimensions time and scan position, similar to the PNPR-CLIM output (see above). For a complete list of geophysical variables accessible to users in HOAPS, see the respective list for level 3 data products in HOAPS in section 5.1 in [D2]. The HOAPS Level 2 data have been generated in the scope of EUMETSAT’s CMSAF and are not part of the C3S portfolio.

Anchor
references references

References

Aires, F., Prigent, C., Rossow, W. B., Rothstein, M. (2001). A new neural network approach including first guess for retrieval of atmospheric water vapor, cloud liquid water path, surface temperature, and emissivities over land from satellite microwave observations. J. Geophys. Res. Atmos., 106, 14887–14907.

...

Related articles