Data assimilation systems are based on methods that combine prior knowledge of the atmosphere (background) with observations in an optimal way taking into account statistical information about the errors of both pieces of information (Kalnay, 2003). Significant improvements in assimilation techniques and numerical weather forecasts in the 1980’s (short range forecast errors of similar magnitude as observation errors) allowed the use of data assimilation systems to provide diagnostic facilities to monitor the quality performance of the observational network (Hollingsworth, et al. 1986). The monitoring of data quality in WDQMS relies on the feedback from several NWP data assimilation systems - mainly the O-B departures. The quality/accuracy indicators to be considered are trueness, precision and gross error (WDQMS Guidance Document). However, for surface observations only trueness has been implemented included in the web tool, whereas for upper-air observations both indicators (Trueness and Precision) are combined 6-hourly and daily aggregations for surface observations, while both trueness and precision are integrated into a single accuracy metric (root mean square error) for upper-air observations. In surface monthly aggregations, the percentage of gross errors for the month is calculated in addition to the accuracy metric root mean square error.
Trueness
The bias (an estimate of systematic error) is used as the measure of trueness (Table 3). The targets regarding trueness are stated so that the bias (average of O-B over a certain period) should be close to zero for all measured variables (sections 2.1.1 and 2.1.2). The trueness is assessed for all the temporal intervals considered in the tool (section 4): 6-hourly, daily and, in the future, monthly. Also, a 5-day moving average (Alert) of the absolute value of daily calculated O-B (Table 9) needs to be calculated daily for all observed variables and compared against the prescribed thresholds (Table 6). This is used as one of the main performance indicators on the daily monitoring activities.
...
Table 6 - Exceedance thresholds of trueness, precision and precision gross errors as defined in the WDQMS Guidance Document (WMO, 2018). Anchor Table6 Table6
Variable | Trueness | Precision | Gross errors |
Surface pressure | 0.5hPa | 1.5hPa | 10hPa |
Geopotential height | 30m | 40m | 100m |
2m Temperature | 0.5K | - | 10K |
10m wind vector | 3.0m/s | 5.0m/s | 15m/s |
10m relative humidity | 10% | - | 30% |
Upper air temperature | 0.5K | 1.5K | 10K |
Upper air wind vector | 3.0m/s | 5.0m/s | 15m/s |
Upper air relative humidity | 10% | - | 30% |
Table 7 - Exceedance limits of measurement uncertainty (combination of trueness and precision) for the relevant surface and upper-air variables as defined by WMO RRR (https://www.wmo-sat.info/oscar/requirements). Anchor Table7 Table7
Variable | Goal | Breakthrough | Threshold |
Surface pressure | 0.5hPa | 1hPa | 1hPa |
Geopotential height | 30m | 30m | 30m |
2m Temperature | 0.5K | 1.0K | 2.0K |
10m wind vector | 0.5m/s | 2.0m/s | 3.0m/s |
10m relative humidity | 2% | 5% | 10% |
Upper air temperature | 0.5K | 1.0K | 3.0 K |
Upper air wind vector | 1.0m/s | 3.0m/s | 5.0m/s |
Upper air relative humidity | 2% | 5% | 10% |
Gross errors
The number of gross errors is obtained by calculating the number of single observations whose O-B departures exceed the prescribed threshold (see Table 6 for all variable thresholds) within the selected period. Then the percentage of gross errors will be calculated as the ratio between the number of gross errors and the total number of single observations for the selected variable and period. The performance threshold for the percentage of gross errors is 15% within a month, which means that the number of gross errors during a month for a particular station and variable should not exceed 15% of all single observations of that particular station and variable. Note that this performance metric is only applied to surface observations in the monthly aggregation.