Periodically Occasionally, we receive reports of negative precipitation totals being computed from IFS output encoded in GRIB. Although such reports often refer to "negative precipitation accumulation", the same issue can affect any field accumulated from the start of the forecast and small but spurious positive accumulations are also possible. Positive accumulations can lead to small increases in, say, solar radiation during night time hours when a zero increase is expected. This page document explains both why this occurs happens and the circumstances in which it occurs.
...
| Note | ||
|---|---|---|
| ||
Note that the problem described here affects only fields accumulated from the start of the forecast. In particular, the ERA5 short forecast accumulations do not suffer from the same problem because they are packed and archived as accumulations since the previous post processing (archiving) step i.e. they are already de-accumulated before packing. |
GRIB packing discretises the values
...
Packing error increases with increasing range of values
The issue for fields accumulated accumulated from the start of the forecast is that the packing parameters - and hence the discretisation and the packing error - change as the range of the values to be packed increases. Because the bits per value remains constant, the effect is to increase the packing error periodically by a factor of two periodically as the range increases. This is illustrated in Figure 1 below which shows how the packing error (the red line) increases as the values of the sunshine duration field increase throughout a 10-day forecast
...
|
|
Figure 2(a): In this case, the packing error=0.5 (represented by the green boxes) and hence hence values are packed to the nearest 1.0. | Figure 2(b): At some point certain specific points as the range of values increases, the packing error increases by a factor of two. In this case, the packing error=1.0 and so values are packed to the nearest 2.0. Similarly, the true input value of 297.4 which yields a decoded value of 297.0 on unpacking in the case shown in Figure 1(a) yields a higher decoded value of 298.0 in the case illustrated in Figure 2(b) |
...
The example below for a small grid of 9 points shows how the packing error and hence the packed values change as the range of the values increases. The deaccumulated packed values are the difference between the packed values at the current step and those at the previous step. Here, a "step" can be considered as any time step where the data has been stored, and hence packed, in GRIB. Typically, this will be a forecast time step in hours. In this example, the values are packed allowing for 8 bits per value.
Step=0
| Accumulated model values | Accumulated packed GRIB values | Deaccumulated packed values | packing error | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
|
- All accumulations start with a zero value at step=0
- The values at all points are stored exactly when packed exactlyin GRIB
Step=1
| Accumulated model values | Accumulated packed GRIB values | Deaccumulated packed values | packing error | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
| 0.03125 |
- The model values increase at all points except the north west
- Not all values can be represented exactly using 8 bits per value
- In particular, the value of 0.031250 at the southern point is exactly the packing error and its packed value becomes 0.062500
- In this case, the packed values are all multiples of 2*packing error=0.0625 so ; model values of 4.55, 5.9 and 2.2 are not stored precisely but only to the nearest multiple of 0.0625.
Step=2
| Accumulated model values | Accumulated packed GRIB values | Deaccumulated packed values | packing error | |||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
| 0.06250 |
...
|
|
Figure 3(a): Illustration showing the change in packing error (y-axis) at increasing lead times (x-axis). The plot shows the correspondence between the lead times where negative precipitation accumulations occur (red points) and the points where the packing error, and hence the discretisation of the field, changes (indicated by the "step" changes in the blue line). The negative accumulations occur due to the subtraction of values from two different levels of descritisation. The example shown is for total precipitation from from a 10-day ENS forecast control run. Refer also to the annotation. | Figure 3(b): Ambiguity in identifying zero rain areas on day 10 in one example ENS forecast control run (see legend).
|
On the map plot example in Figure 3(b) one can see that a large part of the world (blue & pink areas) is affected by the issue. The graph in Figure 3(a) shows how the capacity of the model output to represent small values in 12h (or indeed any) periods diminishes at longer lead times. The discretisation will naturally be worse at the end of, say, the monthly forecast. This also means that the total areas of zero rain in a given period will become larger as the forecast progresses, purely because of the GRIB packing.
For some products or for verification this behaviour is undesirable. The strategy used by ECMWF to overcome this is to set all values in an accumulated field computed by subtraction that are less than a positive threshold , x, to zero. This threshold needs to be large enough to cater for the maximum possible discretization discretisation at the end of the period in question.
Based on the specific analysis presented here, for precipitation totals above 1000mm (from T+0) a discretisation level of ~0.03mm is reached (final step on the example graph). In this case, it is recommended to set the threshold, x, to be, say, 0.04 or 0.08 to cover this. Further analysis of additional forecasts (this particular analysis is based on one month of 00 UTC ENS forecast control runs and where the graph shown represents the worst case scenario among these) these recommendations could be revised. Note also that they apply only to 10 day forecasts.
...