What does decimal precision mean, and how can I use it - ecCodes GRIB FAQ

1.1. What is decimalPrecision?

decimalPrecision is a computed key in ecCodes that controls the number of decimal digits preserved when packing GRIB field data. It provides a convenient way to manage the trade-off between file size and data accuracy without manually calculating bitsPerValue.

1.2. What is the difference between decimalPrecision, setDecimalPrecision, and changeDecimalPrecision?

All three are related but serve different roles:

1.3. How is decimal precision used to calculate bitsPerValue?

The calculation is shown here in a simplified form, without details:

A reference value is subtracted from the data values, and the resulting differences are scaled by 10^D (where D is the decimal precision). The minimum and maximum values are then rounded down to the nearest smaller value, and the difference between them is calculated. Finally, the number of bits required to represent this difference is computed. This number is then assigned to the bitsPerValue  key. The value can then take any value supported by the algorithm. For example, Simple Packing supports 1–56 bits per value, and CCSDS supports 1–32 bits per value.

1.4. How do I use it from the command line?

Use grib_set:

# Repack with 1 decimal digit of precision
grib_set -s setDecimalPrecision=1 input.grib output.grib

# Repack with 3 decimal digits of precision
grib_set -s setDecimalPrecision=3 input.grib output.grib

You can also use the alias:

grib_set -s changeDecimalPrecision=2 input.grib output.grib

1.5. How do I use it from C code?

To set new values:

double values[] = {10.223, 11.244, 9.3708, 8223.1};
size_t values_size = 4;
long decimalPrecision = 2;
codes_set_long(h, "decimalPrecision", decimalPrecision);
codes_set_values(h, "values", values, values_size);

/* bitsPerValue is now automatically calculated */
// long bitsPerValue;
// codes_get_long(h, "bitsPerValue", &bitsPerValue);

To repack existing values:

long decimalPrecision = 2;
codes_set_long(h, "setDecimalPrecision", decimalPrecision);

/* bitsPerValue is now automatically recalculated */
// long bitsPerValue;
// codes_get_long(h, "bitsPerValue", &bitsPerValue);

1.6. How do I use it with grib_filter?

Below are two examples using grib_filter rules files.

Example 1: Setting 4 values and writing with decimal precision

Create a rules file set_values.filter:

# Set 4 data values and pack with 2 decimal digits of precision
set Ni = 2;
set Nj = 2;
set decimalPrecision = 3;
set values = { 2.7047, 2.8053, 2.9012, 3.1109 };
write "out_set_values.grib";

Run it:

grib_filter set_values.filter input.grib

Example 2: Repacking existing values with a new decimal precision

Create a rules file repack.filter:

# Repack the existing data with 1 decimal digit of precision
print "Before: [bitsPerValue=]";
set setDecimalPrecision = 1;
print "After: [bitsPerValue=]";
write "out_repacked.grib";

Run it:

grib_filter repack.filter input.grib

1.7. Does setting changeDecimalPrecision reduce file size?

Typically yes. By specifying fewer decimal digits, the range of packed integers shrinks, and the library can use fewer bitsPerValue. For example, a field originally packed with 16 bits per value may only need 12 bits after setting changeDecimalPrecision=2, reducing the data section size by 25%.

This also has a particularly strong effect on many lossless compression algorithms, for example, with AEC/CCSDS. With a lower decimal precision, the least significant bits (LSBs) are removed. These bits often have low entropy and can degrade compression performance, so removing them improves compression efficiency.

The exact savings depend on the data range and the requested precision.

1.8. Will my data values change after setting setDecimalPrecision?

Yes. Setting setDecimalPrecision triggers a repack cycle. The values are decoded and then re-encoded with the new precision, so there will be a small rounding error limited by the requested number of decimal digits.

For example, with setDecimalPrecision=2, the maximum absolute error will generally not exceed 0.01 (one unit in the second decimal place).

1.9. What happens with constant fields (all values identical)?

Constant fields are handled as a special case. When all values are the same, bitsPerValue is set to 0 regardless of the decimal precision.

1.10. Can I use setDecimalPrecision with any packing type?

setDecimalPrecision works with algorithms that support decimal scaling, e.g.,  Simple Packing and CCSDS. Care should be taken when using it with spectral data or complex packing types, as interactions between the decimal scale factor and certain packing algorithms may produce unexpected results.

TODO (maee): List supported algorithms

1.11. Can I set decimalPrecision directly instead of setDecimalPrecision?

Always use setDecimalPrecision (or its alias changeDecimalPrecision) to ensure the data is properly repacked.

1.12. What is the relationship between decimalPrecision and bitsPerValue?

There are two different ways to control packing accuracy: