Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The issue for accumulated fields is that the packing parameters - and hence the discretisation and the packing error - change as the range of values increases.  Because the bits per value remains constant, the effect is to increase the packing error by a factor of two periodically as the range increases.  This is illustrated in the figure below which shows how the packing error (the red line) increases as the values of the sunshine duration field increase throughout a 10-day forecast

Image Modified

The plot on the left shows how the packing error (red line) increases as the range of values of the sunshine duration field (blue line) increases throughout at 10-day forecast.

Each time the range of values increases by a factor of two (indicated by the points where the blue line crosses the horizontal dashed lines) the packing error also increases by a factor of two.

Although the packing error is less than 1 second in the first 24 hours of the forecast, it is 8 seconds by the end of the forecast.


Packed values depend on the packing error

The increase in packing error also has an effect on the packed values themselves.  This is illustrated in the diagrams below.

Image Modified

Image Modified

(a) In this case, the packing error=0.5 (represented by the green boxes) and hence  values are packed to the nearest 1.0.

Note that the true "exact model" values represented at the top are not stored exactly.  The value 291.7 yields the decoded value 292.0 on unpacking whereas 297.4 becomes 297.0 on unpacking. 

(b) At some point as the range of values increases, the packing error increases by a factor of two.  In this case, the packing error=1.0 and so values are packed to the nearest 2.0. 

Here we see that a true input value of 294.6 which yields a decoded value of 295.0 on unpacking in case (a) yields the lower decoded value of 294.0 even though the true input value is unchanged.

Similarly, the true input value of 297.4 which yields a decoded value of 297.0 on unpacking in case (a) yields a higher decoded value of 298.0 in case (b)


How small spurious negative - and positive - values appear when de-accumulating an accumulated field

The example below for a small grid of 9 points shows how the packing error and hence the packed values change as the range of the values increases.

Step=0

Accumulated
model
values
Accumulated
packed
GRIB values
Deaccumulated
packed values
packing error


0.000000.000000.00000
0.000000.000000.00000
0.000000.000000.00000



0.000000.000000.00000
0.000000.000000.00000
0.000000.000000.00000





  • All accumulations start with a zero value at step=0
  • The values at all points are packed exactly


Step=1

Accumulated
model
values
Accumulated
packed
GRIB values
Deaccumulated
packed values
packing error


0.000001.5000010.00000
3.250004.550005.90000
2.200000.031252.00000



0.000001.5000010.00000
3.250004.562505.87500
2.187500.062502.00000



0.000001.5000010.00000
3.250004.562505.87500
2.187500.062502.00000


0.03125
  • The model values increase at all points except the north west
  • Not all values can be represented exactly using 8 bits per value
  • In particular, the value of 0.031250 at the southern point is exactly the packing error and its packed value becomes 0.062500
  • In this case, the packed values are all multiples of 2*packing error=0.0625 so values of 4.55, 5.9 and 2.2 are not stored precisely

Step=2

Accumulated
model
values
Accumulated
packed
GRIB values
Deaccumulated
packed values
packing error


0.000001.5000020.00000
3.250004.550005.90000
2.200000.031252.00000



0.000001.5000020.00000
3.250004.500005.87500
2.250000.000002.00000



0.000000.0000010.00000
0.00000-0.062500.00000
0.06250-0.062500.00000


0.06250
  • Only the model value at the north east point increases
  • The increase in the range of values increases the packing error by a factor of 2

  • Now all packed values are multiples of 2*packing error=0.125

  • The value at the central point of 4.55 is now packed as 4.5 compared to a packed value of 4.5625 at the previous step - a negative accumulation of -0.0625
  • The value at the south west point of 2.2 is now packed as 2.25 compared to a packed value of 2.1857 at the previous step - a positive accumulation of +0.0625

Step=3

Accumulated
model
values
Accumulated
packed
GRIB values
Deaccumulated
packed values
packing error


0.000001.5000040.00000
3.250004.550005.90000
2.200000.031252.00000



0.000001.5000040.00000
3.250004.500006.00000
2.250000.000002.00000



0.000000.0000020.00000
0.000000.000000.12500
0.000000.000000.00000


0.12500
  • The model value at the north east point increases further to 40.0
  • Model values at all other points remain the same as at step=3
  • The increase in the range of values increases the packing error.
  • Now all packed values are multiples of 2*packing error=0.25
  • The model value at the eastern point is unchanged at 5.9 but now its packed value is 6.0 giving an apparent positive accumulation  of +0.125

...

For accumulated fields, such as precipitation, the GRIB packing can have a strange affect on the totals, even when the data are used on the native model reduced Gaussian grid.  In some cases, the values concerned are sufficiently small that the issue can be ignored, but for others - for example when looking at the frequency of zero rain, or light drizzle on forecast day 10 - action is needed. The figures below highlight the issue is and its impacts:

Image Modified

Image Modified

(a) Illustration of discretisation levels in 12h precipitation totals calculated for one ENS forecast control run - refer also to annotation

(b) Ambiguity in identifying zero rain areas on day 10 in one example ENS forecast control run (see legend)

On the map plot example (b) one can see that a large part of the world (blue & pink areas) is affected by the issue.  The graph (a) shows how the capacity of the model output to represent small values in 12h (or indeed any) periods diminishes at longer lead times. The discretisation will naturally be worse at the end of, say, the monthly forecast. This also means that the total areas of zero rain in a given period will become larger as the forecast progresses, purely because of the GRIB packing.

...

Content by Label
showLabelsfalse
max5
spacesUDOC
showSpacefalse
sortmodified
reversetrue
typepage
excludeCurrenttrue
cqllabel in ("grib","packing") and type = "page" and space = "UDOC"
labelspacking grib

...

Page properties
hiddentrue


Related issues