Introduction
The new CDS and ADS systems (currently the beta system) have a new GRIB to netCDF converter. This new converter is based on xarray using the cfgrib engine. As each dataset has it's own particular features, the configuration used differs from dataset to dataset to ensure that the returned results are correct and appropriately strcutured.
The general workflow of the new conversion is as follows (for a more detailed description, expand the Jupyter Notebook below):
- Open the GRIB file as an xarray.Dataset using the cfgrib engine
- Where necessary this is a list of xarray.Dataset such that the data is organised into complete and compatible hypercubes
- As each dataset has it's own specifics, the options used when opening the GRIB files differ from dataset to dataset. These options can be made available upon request, but an understanding of the Jupyter Notebook below is expected should users wish to make use of them.
- Rename dimension/coordinate variables to match those described in New Coordinate variable names section
- Additionally, dimensions are expanded to ensure that certain variables are also dimensions of the dataset, e.g. the valid_time dimension when there is a single time step.
- Additionally, dimensions are expanded to ensure that certain variables are also dimensions of the dataset, e.g. the valid_time dimension when there is a single time step.
- Store as netCDF4 file[s] (with internal compression) and return to the user
As GRIB is the native format of most of the datasets produced by ECMWF, the format used in the netCDF files served by the CDS/ADS should not be considered a long-term stable format. Therefore, it is highly discouraged that these netCDF files are used in operational/downstream services as the standard is subject to change. |
There are other openly available tools and software available for working with GRIB files, including conversion to netCDF. Please see the following resources for more guidance on using the GRIB files produce by ECMWF directly:
- earthkit is ECMWF supported open source python software which simplifies handling of GRIB files: https://earthkit.readthedocs.io/en/latest/
- eccodes is a package developed by ECMWF that includes libraries and binaries for reading GRIB files, it also includes the "grib_to_netcdf" executable that was used by the legacy ADS/CDS, but please note that this executable is coming to the end of supported period:
- brew install for Mac: https://formulae.brew.sh/formula/eccodes
- conda-forge install for conda environments: https://anaconda.org/conda-forge/eccodes
- manual install instructions: ecCodes installation
- cfgrib is an engine for opening GRIB files in xarray objects: https://pypi.org/project/cfgrib/0.8.4.5/
Why we are changing
There are a number of important reasons for the change in the netCDF produced by the CDS-Beta.
- NetCDF4 is a more modern, more capable, and more future-proof version of netCDF.
- The legacy CDS and ADS used a mixture of 'grib_to_netcdf' (netCDF3) and cfgrib (netCDF4) to convert GRIB files to netCDF, rather than a single common form of netCDF.
- Additionally if files were post-processed in the CDS toolbox (e.g. daily statistics), they were also converted to the common data model (CDS-CDM), which introduced additional differences in the netCDF structure and content.
- As such, the legacy CDS and ADS system supported 3 different netCDF conversions and standards, all of which had their own issues.
The new systems will use cfgrib for the conversion, with some modifications as documented below. The same converter is used for direct downloads and post-processed data (e.g. the daily statistics from ERA5 and ERA5-Land), hence users will have consistency in the various netCDF files received from the modernised (currently beta) portals.
It is important to understand that given the intrinsic differences between GRIB and netCDF there is not a one size fits all approach when converting GRIB to netCDF. We aim to provide users with some sensible default options, but given the code used to perform the conversion is open source and documented it is possible for users to fine tune the conversion to their requirements if needed.
Please also note that these changes may mean that some software packages (e,g, GrADS, CDO) may not be able to open these new netCDF files without modification (e.g. changes to dimension order, dimension names etc.) |
Jupter Notebook demonstration
Github repository: Grib_to_netcdf |
Summary of changes
This section summarises the changes being made for the various dataset families that are effected.
ERA5
Legacy converter
netcdf era5old {
dimensions:
longitude = 1440 ;
latitude = 721 ;
time = 1 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int time(time) ;
time:units = "hours since 1900-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short u10(time, latitude, longitude) ;
u10:scale_factor = 0.00077681819178887 ;
u10:add_offset = -0.572623516517769 ;
u10:_FillValue = -32767s ;
u10:missing_value = -32767s ;
u10:units = "m s**-1" ;
u10:long_name = "10 metre U wind component" ;
// global attributes:
:Conventions = "CF-1.6" ;
:history = "2024-09-06 18:44:53 GMT by grib_to_netcdf-2.28.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data0/adaptor.mars.internal-1725648293.1768346-3616-5-5f732e74-655a-44df-bade-83de1e10126d.nc /cache/tmp/5f732e74-655a-44df-bade-83de1e10126d-adaptor.mars.internal-1725648291.279607-3616-1-tmp.grib" ;
} |
Main differences
|
New converter
netcdf era5new {
dimensions:
valid_time = 1 ;
latitude = 721 ;
longitude = 1440 ;
variables:
int64 number ;
string number:long_name = "ensemble member numerical id" ;
string number:units = "1" ;
string number:standard_name = "realization" ;
int64 valid_time(valid_time) ;
string valid_time:long_name = "time" ;
string valid_time:standard_name = "time" ;
string valid_time:units = "seconds since 1970-01-01" ;
string valid_time:calendar = "proleptic_gregorian" ;
double latitude(latitude) ;
latitude:_FillValue = NaN ;
string latitude:units = "degrees_north" ;
string latitude:standard_name = "latitude" ;
string latitude:long_name = "latitude" ;
string latitude:stored_direction = "decreasing" ;
double longitude(longitude) ;
longitude:_FillValue = NaN ;
string longitude:units = "degrees_east" ;
string longitude:standard_name = "longitude" ;
string longitude:long_name = "longitude" ;
string expver ;
float u10(valid_time, latitude, longitude) ;
u10:_FillValue = NaNf ;
u10:GRIB_paramId = 165LL ;
string u10:GRIB_dataType = "an" ;
u10:GRIB_numberOfPoints = 1038240LL ;
string u10:GRIB_typeOfLevel = "surface" ;
u10:GRIB_stepUnits = 1LL ;
string u10:GRIB_stepType = "instant" ;
string u10:GRIB_gridType = "regular_ll" ;
u10:GRIB_uvRelativeToGrid = 0LL ;
u10:GRIB_NV = 0LL ;
u10:GRIB_Nx = 1440LL ;
u10:GRIB_Ny = 721LL ;
string u10:GRIB_cfName = "unknown" ;
string u10:GRIB_cfVarName = "u10" ;
string u10:GRIB_gridDefinitionDescription = "Latitude/Longitude Grid" ;
u10:GRIB_iDirectionIncrementInDegrees = 0.25 ;
u10:GRIB_iScansNegatively = 0LL ;
u10:GRIB_jDirectionIncrementInDegrees = 0.25 ;
u10:GRIB_jPointsAreConsecutive = 0LL ;
u10:GRIB_jScansPositively = 0LL ;
u10:GRIB_latitudeOfFirstGridPointInDegrees = 90. ;
u10:GRIB_latitudeOfLastGridPointInDegrees = -90. ;
u10:GRIB_longitudeOfFirstGridPointInDegrees = -180. ;
u10:GRIB_longitudeOfLastGridPointInDegrees = 179.75 ;
u10:GRIB_missingValue = 3.40282346638529e+38 ;
string u10:GRIB_name = "10 metre U wind component" ;
string u10:GRIB_shortName = "10u" ;
u10:GRIB_totalNumber = 0LL ;
string u10:GRIB_units = "m s**-1" ;
string u10:long_name = "10 metre U wind component" ;
string u10:units = "m s**-1" ;
string u10:standard_name = "unknown" ;
u10:GRIB_surface = 0. ;
string u10:coordinates = "number valid_time latitude longitude expver" ;
// global attributes:
string :GRIB_centre = "ecmf" ;
string :GRIB_centreDescription = "European Centre for Medium-Range Weather Forecasts" ;
:GRIB_subCentre = 0LL ;
string :Conventions = "CF-1.7" ;
string :institution = "European Centre for Medium-Range Weather Forecasts" ;
string :history = "2024-09-06T16:33 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {\"source\": \"data.grib\", \"filter_by_keys\": {\"stream\": [\"oper\"]}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
ERA5-Land
Legacy converter
netcdf era5landold {
dimensions:
longitude = 3600 ;
latitude = 1801 ;
time = 1 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int time(time) ;
time:units = "hours since 1900-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short d2m(time, latitude, longitude) ;
d2m:scale_factor = 0.0014517375787771 ;
d2m:add_offset = 255.978186484726 ;
d2m:_FillValue = -32767s ;
d2m:missing_value = -32767s ;
d2m:units = "K" ;
d2m:long_name = "2 metre dewpoint temperature" ;
// global attributes:
:Conventions = "CF-1.6" ;
:history = "2024-09-06 19:43:33 GMT by grib_to_netcdf-2.28.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data0/adaptor.mars.internal-1725651812.3077197-6335-1-ac0ec060-3156-4241-afb4-43a4e1d5ab1a.nc /cache/tmp/ac0ec060-3156-4241-afb4-43a4e1d5ab1a-adaptor.mars.internal-1725651811.1974177-6335-1-tmp.grib" ;
} |
Main differences
|
New converter
netcdf era5landnew {
dimensions:
valid_time = 1 ;
latitude = 1801 ;
longitude = 3600 ;
variables:
int64 number ;
string number:long_name = "ensemble member numerical id" ;
string number:units = "1" ;
string number:standard_name = "realization" ;
int64 valid_time(valid_time) ;
string valid_time:long_name = "time" ;
string valid_time:standard_name = "time" ;
string valid_time:units = "seconds since 1970-01-01" ;
string valid_time:calendar = "proleptic_gregorian" ;
double latitude(latitude) ;
latitude:_FillValue = NaN ;
string latitude:units = "degrees_north" ;
string latitude:standard_name = "latitude" ;
string latitude:long_name = "latitude" ;
string latitude:stored_direction = "decreasing" ;
double longitude(longitude) ;
longitude:_FillValue = NaN ;
string longitude:units = "degrees_east" ;
string longitude:standard_name = "longitude" ;
string longitude:long_name = "longitude" ;
string expver ;
float d2m(valid_time, latitude, longitude) ;
d2m:_FillValue = NaNf ;
d2m:GRIB_paramId = 168LL ;
string d2m:GRIB_dataType = "fc" ;
d2m:GRIB_numberOfPoints = 6483600LL ;
string d2m:GRIB_typeOfLevel = "surface" ;
d2m:GRIB_stepUnits = 1LL ;
string d2m:GRIB_stepType = "instant" ;
string d2m:GRIB_gridType = "regular_ll" ;
d2m:GRIB_uvRelativeToGrid = 0LL ;
d2m:GRIB_NV = 0LL ;
d2m:GRIB_Nx = 3600LL ;
d2m:GRIB_Ny = 1801LL ;
string d2m:GRIB_cfName = "unknown" ;
string d2m:GRIB_cfVarName = "d2m" ;
string d2m:GRIB_gridDefinitionDescription = "Latitude/Longitude Grid" ;
d2m:GRIB_iDirectionIncrementInDegrees = 0.1 ;
d2m:GRIB_iScansNegatively = 0LL ;
d2m:GRIB_jDirectionIncrementInDegrees = 0.1 ;
d2m:GRIB_jPointsAreConsecutive = 0LL ;
d2m:GRIB_jScansPositively = 0LL ;
d2m:GRIB_latitudeOfFirstGridPointInDegrees = 90. ;
d2m:GRIB_latitudeOfLastGridPointInDegrees = -90. ;
d2m:GRIB_longitudeOfFirstGridPointInDegrees = 0. ;
d2m:GRIB_longitudeOfLastGridPointInDegrees = 359.9 ;
d2m:GRIB_missingValue = 3.40282346638529e+38 ;
string d2m:GRIB_name = "2 metre dewpoint temperature" ;
string d2m:GRIB_shortName = "2d" ;
d2m:GRIB_totalNumber = 0LL ;
string d2m:GRIB_units = "K" ;
string d2m:long_name = "2 metre dewpoint temperature" ;
string d2m:units = "K" ;
string d2m:standard_name = "unknown" ;
d2m:GRIB_surface = 0. ;
string d2m:coordinates = "number valid_time latitude longitude expver" ;
// global attributes:
string :GRIB_centre = "ecmf" ;
string :GRIB_centreDescription = "European Centre for Medium-Range Weather Forecasts" ;
:GRIB_subCentre = 0LL ;
string :Conventions = "CF-1.7" ;
string :institution = "European Centre for Medium-Range Weather Forecasts" ;
string :history = "2024-09-06T16:34 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {\"source\": \"data.grib\", \"filter_by_keys\": {}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
Seasonal Forecasts
Legacy converter
netcdf seasonalforecastsold {
dimensions:
longitude = 360 ;
latitude = 180 ;
number = 51 ;
time = 1 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int number(number) ;
number:long_name = "ensemble_member" ;
int time(time) ;
time:units = "hours since 1900-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short u10(time, number, latitude, longitude) ;
u10:scale_factor = 0.000725178414927284 ;
u10:add_offset = -0.292896463719182 ;
u10:_FillValue = -32767s ;
u10:missing_value = -32767s ;
u10:units = "m s**-1" ;
u10:long_name = "10 metre U wind component" ;
// global attributes:
:Conventions = "CF-1.6" ;
:history = "2024-09-06 17:23:03 GMT by grib_to_netcdf-2.28.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf -S param -o /cache/data3/adaptor.mars.external-1725643382.8773675-11850-19-c9a59396-8d1c-4959-b65c-f1b44cac2259.nc /cache/tmp/c9a59396-8d1c-4959-b65c-f1b44cac2259-adaptor.mars.external-1725643380.1017525-11850-5-tmp.grib" ;
} |
Main differences
|
New converter
netcdf seasonalforecastnew {
dimensions:
number = 51 ;
forecast_reference_time = 1 ;
forecast_period = 1 ;
latitude = 180 ;
longitude = 360 ;
variables:
int64 number(number) ;
string number:long_name = "ensemble member numerical id" ;
string number:units = "1" ;
string number:standard_name = "realization" ;
int64 forecast_reference_time(forecast_reference_time) ;
string forecast_reference_time:long_name = "initial time of forecast" ;
string forecast_reference_time:standard_name = "forecast_reference_time" ;
string forecast_reference_time:units = "seconds since 1970-01-01" ;
string forecast_reference_time:calendar = "proleptic_gregorian" ;
double forecast_period(forecast_period) ;
forecast_period:_FillValue = NaN ;
string forecast_period:long_name = "time since forecast_reference_time" ;
string forecast_period:standard_name = "forecast_period" ;
string forecast_period:units = "hours" ;
double latitude(latitude) ;
latitude:_FillValue = NaN ;
string latitude:units = "degrees_north" ;
string latitude:standard_name = "latitude" ;
string latitude:long_name = "latitude" ;
string latitude:stored_direction = "decreasing" ;
double longitude(longitude) ;
longitude:_FillValue = NaN ;
string longitude:units = "degrees_east" ;
string longitude:standard_name = "longitude" ;
string longitude:long_name = "longitude" ;
double valid_time ;
valid_time:_FillValue = NaN ;
string valid_time:standard_name = "time" ;
string valid_time:long_name = "time" ;
string valid_time:units = "seconds since 1970-01-01" ;
string valid_time:calendar = "proleptic_gregorian" ;
float u10(number, forecast_reference_time, forecast_period, latitude, longitude) ;
u10:_FillValue = NaNf ;
u10:GRIB_paramId = 165LL ;
string u10:GRIB_dataType = "fc" ;
u10:GRIB_numberOfPoints = 64800LL ;
string u10:GRIB_typeOfLevel = "surface" ;
u10:GRIB_stepUnits = 1LL ;
string u10:GRIB_stepType = "instant" ;
string u10:GRIB_gridType = "regular_ll" ;
u10:GRIB_uvRelativeToGrid = 0LL ;
u10:GRIB_NV = 0LL ;
u10:GRIB_Nx = 360LL ;
u10:GRIB_Ny = 180LL ;
string u10:GRIB_cfName = "unknown" ;
string u10:GRIB_cfVarName = "u10" ;
string u10:GRIB_gridDefinitionDescription = "Latitude/Longitude Grid" ;
u10:GRIB_iDirectionIncrementInDegrees = 1. ;
u10:GRIB_iScansNegatively = 0LL ;
u10:GRIB_jDirectionIncrementInDegrees = 1. ;
u10:GRIB_jPointsAreConsecutive = 0LL ;
u10:GRIB_jScansPositively = 0LL ;
u10:GRIB_latitudeOfFirstGridPointInDegrees = 89.5 ;
u10:GRIB_latitudeOfLastGridPointInDegrees = -89.5 ;
u10:GRIB_longitudeOfFirstGridPointInDegrees = 0.5 ;
u10:GRIB_longitudeOfLastGridPointInDegrees = 359.5 ;
u10:GRIB_missingValue = 3.40282346638529e+38 ;
string u10:GRIB_name = "10 metre U wind component" ;
string u10:GRIB_shortName = "10u" ;
u10:GRIB_system = 51LL ;
u10:GRIB_totalNumber = 0LL ;
string u10:GRIB_units = "m s**-1" ;
string u10:long_name = "10 metre U wind component" ;
string u10:units = "m s**-1" ;
string u10:standard_name = "unknown" ;
u10:GRIB_surface = 0. ;
string u10:coordinates = "number time step latitude longitude valid_time" ;
// global attributes:
:GRIB_edition = 1LL ;
string :GRIB_centre = "ecmf" ;
string :GRIB_centreDescription = "European Centre for Medium-Range Weather Forecasts" ;
:GRIB_subCentre = 0LL ;
string :Conventions = "CF-1.7" ;
string :institution = "European Centre for Medium-Range Weather Forecasts" ;
string :history = "2024-09-06T16:54 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {\"source\": \"data.grib\", \"filter_by_keys\": {\"centre\": \"ecmf\"}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
CARRA/CERRA/UERRA
Legacy converter
netcdf carraold {
dimensions:
y = 989 ;
x = 789 ;
variables:
int64 time ;
time:long_name = "initial time of forecast" ;
time:standard_name = "forecast_reference_time" ;
time:units = "seconds since 1970-01-01" ;
time:calendar = "proleptic_gregorian" ;
double step ;
step:_FillValue = NaN ;
step:long_name = "time since forecast_reference_time" ;
step:standard_name = "forecast_period" ;
step:units = "hours" ;
double heightAboveGround ;
heightAboveGround:_FillValue = NaN ;
heightAboveGround:long_name = "height above the surface" ;
heightAboveGround:units = "m" ;
heightAboveGround:positive = "up" ;
heightAboveGround:standard_name = "height" ;
double latitude(y, x) ;
latitude:_FillValue = NaN ;
latitude:units = "degrees_north" ;
latitude:standard_name = "latitude" ;
latitude:long_name = "latitude" ;
double longitude(y, x) ;
longitude:_FillValue = NaN ;
longitude:units = "degrees_east" ;
longitude:standard_name = "longitude" ;
longitude:long_name = "longitude" ;
double valid_time ;
valid_time:_FillValue = NaN ;
valid_time:standard_name = "time" ;
valid_time:long_name = "time" ;
valid_time:units = "seconds since 1970-01-01" ;
valid_time:calendar = "proleptic_gregorian" ;
float efg10(y, x) ;
efg10:_FillValue = NaNf ;
efg10:GRIB_paramId = 260646LL ;
efg10:GRIB_dataType = "fc" ;
efg10:GRIB_numberOfPoints = 780321LL ;
efg10:GRIB_typeOfLevel = "heightAboveGround" ;
efg10:GRIB_stepUnits = 1LL ;
efg10:GRIB_stepType = "max" ;
efg10:GRIB_gridType = "lambert" ;
efg10:GRIB_DxInMetres = 2500. ;
efg10:GRIB_DyInMetres = 2500. ;
efg10:GRIB_LaDInDegrees = 80. ;
efg10:GRIB_Latin1InDegrees = 80. ;
efg10:GRIB_Latin2InDegrees = 80. ;
efg10:GRIB_LoVInDegrees = 326. ;
efg10:GRIB_NV = 0LL ;
efg10:GRIB_Nx = 789LL ;
efg10:GRIB_Ny = 989LL ;
efg10:GRIB_cfName = "unknown" ;
efg10:GRIB_cfVarName = "efg10" ;
efg10:GRIB_gridDefinitionDescription = "Lambert conformal" ;
efg10:GRIB_iScansNegatively = 0LL ;
efg10:GRIB_jPointsAreConsecutive = 0LL ;
efg10:GRIB_jScansPositively = 1LL ;
efg10:GRIB_latitudeOfFirstGridPointInDegrees = 70.135 ;
efg10:GRIB_latitudeOfSouthernPoleInDegrees = 0. ;
efg10:GRIB_longitudeOfFirstGridPointInDegrees = 340.592 ;
efg10:GRIB_longitudeOfSouthernPoleInDegrees = 0. ;
efg10:GRIB_missingValue = 9999LL ;
efg10:GRIB_name = "10 metre eastward wind gust since previous post-processing" ;
efg10:GRIB_shortName = "10efg" ;
efg10:GRIB_units = "m s**-1" ;
efg10:long_name = "10 metre eastward wind gust since previous post-processing" ;
efg10:units = "m s**-1" ;
efg10:standard_name = "unknown" ;
efg10:coordinates = "time step heightAboveGround latitude longitude valid_time" ;
// global attributes:
:GRIB_edition = 2LL ;
:GRIB_centre = "enmi" ;
:GRIB_centreDescription = "Oslo" ;
:GRIB_subCentre = 255LL ;
:Conventions = "CF-1.7" ;
:institution = "Oslo" ;
:history = "2024-09-06T16:40 GRIB to CDM+CF via cfgrib-0.9.9.1/ecCodes-2.34.1 with {\"source\": \"/cache/tmp/21b28627-bebd-4bda-9a80-776b3885380c-adaptor.mars.external-1725640741.1036253-24617-1-tmp.grib\", \"filter_by_keys\": {}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
Main differences
|
New converter
netcdf carranew {
dimensions:
valid_time = 1 ;
y = 989 ;
x = 789 ;
variables:
int64 valid_time(valid_time) ;
string valid_time:long_name = "time" ;
string valid_time:standard_name = "time" ;
string valid_time:units = "seconds since 1970-01-01" ;
string valid_time:calendar = "proleptic_gregorian" ;
double latitude(y, x) ;
latitude:_FillValue = NaN ;
string latitude:units = "degrees_north" ;
string latitude:standard_name = "latitude" ;
string latitude:long_name = "latitude" ;
double longitude(y, x) ;
longitude:_FillValue = NaN ;
string longitude:units = "degrees_east" ;
string longitude:standard_name = "longitude" ;
string longitude:long_name = "longitude" ;
string expver ;
float efg10(valid_time, y, x) ;
efg10:_FillValue = NaNf ;
efg10:GRIB_paramId = 260646LL ;
string efg10:GRIB_dataType = "fc" ;
efg10:GRIB_numberOfPoints = 780321LL ;
string efg10:GRIB_typeOfLevel = "heightAboveGround" ;
efg10:GRIB_stepUnits = 1LL ;
string efg10:GRIB_stepType = "max" ;
string efg10:GRIB_gridType = "lambert" ;
efg10:GRIB_uvRelativeToGrid = 1LL ;
efg10:GRIB_DxInMetres = 2500. ;
efg10:GRIB_DyInMetres = 2500. ;
efg10:GRIB_LaDInDegrees = 80. ;
efg10:GRIB_Latin1InDegrees = 80. ;
efg10:GRIB_Latin2InDegrees = 80. ;
efg10:GRIB_LoVInDegrees = 326. ;
efg10:GRIB_NV = 0LL ;
efg10:GRIB_Nx = 789LL ;
efg10:GRIB_Ny = 989LL ;
string efg10:GRIB_cfName = "unknown" ;
string efg10:GRIB_cfVarName = "efg10" ;
string efg10:GRIB_gridDefinitionDescription = "Lambert conformal" ;
efg10:GRIB_iScansNegatively = 0LL ;
efg10:GRIB_jPointsAreConsecutive = 0LL ;
efg10:GRIB_jScansPositively = 1LL ;
efg10:GRIB_latitudeOfFirstGridPointInDegrees = 70.135 ;
efg10:GRIB_latitudeOfSouthernPoleInDegrees = 0. ;
efg10:GRIB_longitudeOfFirstGridPointInDegrees = 340.592 ;
efg10:GRIB_longitudeOfSouthernPoleInDegrees = 0. ;
efg10:GRIB_missingValue = 3.40282346638529e+38 ;
string efg10:GRIB_name = "10 metre eastward wind gust since previous post-processing" ;
string efg10:GRIB_shortName = "10efg" ;
string efg10:GRIB_units = "m s**-1" ;
string efg10:long_name = "10 metre eastward wind gust since previous post-processing" ;
string efg10:units = "m s**-1" ;
string efg10:standard_name = "unknown" ;
efg10:GRIB_heightAboveGround = 10. ;
string efg10:coordinates = "valid_time latitude longitude expver" ;
// global attributes:
string :GRIB_centre = "enmi" ;
string :GRIB_centreDescription = "Oslo" ;
:GRIB_subCentre = 255LL ;
string :Conventions = "CF-1.7" ;
string :institution = "Oslo" ;
string :history = "2024-09-06T16:37 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {\"source\": \"data.grib\", \"filter_by_keys\": {}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
CAMS global atmospheric composition forecasts
Legacy converter
netcdf camsglobalold {
dimensions:
longitude = 900 ;
latitude = 451 ;
level = 3 ;
time = 2 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int level(level) ;
level:long_name = "model_level_number" ;
int time(time) ;
time:units = "hours since 1900-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short aermr04(time, level, latitude, longitude) ;
aermr04:scale_factor = 4.85165910314615e-22 ;
aermr04:add_offset = 1.33913858161131e-16 ;
aermr04:_FillValue = -32767s ;
aermr04:missing_value = -32767s ;
aermr04:units = "kg kg**-1" ;
aermr04:long_name = "Dust Aerosol (0.03 - 0.55 um) Mixing Ratio" ;
// global attributes:
:Conventions = "CF-1.6" ;
:history = "2024-09-06 16:44:58 GMT by grib_to_netcdf-2.25.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf.bin -S param -o /cache/tmp/b29a7c5c-7228-49d6-a532-d3e3c5ae6f22-adaptor.mars_constrained.internal-1725641098.4198422-8971-22-tmp.nc /cache/tmp/b29a7c5c-7228-49d6-a532-d3e3c5ae6f22-adaptor.mars_constrained.internal-1725641098.34029-8971-21-tmp.grib" ;
} |
Main differences
|
New converter
netcdf camsglobalnew {
dimensions:
forecast_period = 1 ;
forecast_reference_time = 2 ;
model_level = 3 ;
latitude = 451 ;
longitude = 900 ;
variables:
double forecast_period(forecast_period) ;
forecast_period:_FillValue = NaN ;
string forecast_period:long_name = "time since forecast_reference_time" ;
string forecast_period:standard_name = "forecast_period" ;
string forecast_period:units = "hours" ;
int64 forecast_reference_time(forecast_reference_time) ;
string forecast_reference_time:long_name = "initial time of forecast" ;
string forecast_reference_time:standard_name = "forecast_reference_time" ;
string forecast_reference_time:units = "seconds since 1970-01-01" ;
string forecast_reference_time:calendar = "proleptic_gregorian" ;
double model_level(model_level) ;
model_level:_FillValue = NaN ;
string model_level:long_name = "hybrid level" ;
string model_level:units = "1" ;
string model_level:positive = "down" ;
string model_level:standard_name = "atmosphere_hybrid_sigma_pressure_coordinate" ;
double latitude(latitude) ;
latitude:_FillValue = NaN ;
string latitude:units = "degrees_north" ;
string latitude:standard_name = "latitude" ;
string latitude:long_name = "latitude" ;
string latitude:stored_direction = "decreasing" ;
double longitude(longitude) ;
longitude:_FillValue = NaN ;
string longitude:units = "degrees_east" ;
string longitude:standard_name = "longitude" ;
string longitude:long_name = "longitude" ;
double valid_time(forecast_reference_time, forecast_period) ;
valid_time:_FillValue = NaN ;
string valid_time:standard_name = "time" ;
string valid_time:long_name = "time" ;
string valid_time:units = "seconds since 1970-01-01T00:00:00" ;
string valid_time:calendar = "proleptic_gregorian" ;
float aermr04(forecast_period, forecast_reference_time, model_level, latitude, longitude) ;
aermr04:_FillValue = NaNf ;
aermr04:GRIB_paramId = 210004LL ;
string aermr04:GRIB_dataType = "fc" ;
aermr04:GRIB_numberOfPoints = 405900LL ;
string aermr04:GRIB_typeOfLevel = "hybrid" ;
aermr04:GRIB_stepUnits = 1LL ;
string aermr04:GRIB_stepType = "instant" ;
string aermr04:GRIB_gridType = "regular_ll" ;
aermr04:GRIB_uvRelativeToGrid = 0LL ;
aermr04:GRIB_NV = 276LL ;
aermr04:GRIB_Nx = 900LL ;
aermr04:GRIB_Ny = 451LL ;
string aermr04:GRIB_cfName = "unknown" ;
string aermr04:GRIB_cfVarName = "aermr04" ;
string aermr04:GRIB_gridDefinitionDescription = "Latitude/longitude" ;
aermr04:GRIB_iDirectionIncrementInDegrees = 0.4 ;
aermr04:GRIB_iScansNegatively = 0LL ;
aermr04:GRIB_jDirectionIncrementInDegrees = 0.4 ;
aermr04:GRIB_jPointsAreConsecutive = 0LL ;
aermr04:GRIB_jScansPositively = 0LL ;
aermr04:GRIB_latitudeOfFirstGridPointInDegrees = 90. ;
aermr04:GRIB_latitudeOfLastGridPointInDegrees = -90. ;
aermr04:GRIB_longitudeOfFirstGridPointInDegrees = 0. ;
aermr04:GRIB_longitudeOfLastGridPointInDegrees = 359.6 ;
aermr04:GRIB_missingValue = 3.40282346638529e+38 ;
string aermr04:GRIB_name = "Dust Aerosol (0.03 - 0.55 um) Mixing Ratio" ;
aermr04:GRIB_pv = 0., 2.0003650188446, 3.10224103927612, 4.66608381271362, 6.82797718048096, 9.74696636199951, 13.6054239273071, 18.6089305877686, 24.9857177734375, 32.985710144043, 42.8792419433594, 54.9554634094238, 69.5205764770508, 86.895881652832, 107.415740966797, 131.425506591797, 159.279403686523, 191.338562011719, 227.968948364258, 269.539581298828, 316.420745849609, 368.982360839844, 427.592498779297, 492.616027832031, 564.413452148438, 643.339904785156, 729.744140625, 823.967834472656, 926.344909667969, 1037.201171875, 1156.85363769531, 1285.6103515625, 1423.77014160156, 1571.62292480469, 1729.44897460938, 1897.51928710938, 2076.09594726562, 2265.431640625, 2465.7705078125, 2677.34814453125, 2900.39135742188, 3135.11938476562, 3381.74365234375, 3640.46826171875, 3911.49047851562, 4194.9306640625, 4490.8173828125, 4799.1494140625, 5119.89501953125, 5452.99072265625, 5798.3447265625, 6156.07421875, 6526.94677734375, 6911.87060546875, 7311.869140625, 7727.412109375, 8159.35400390625, 8608.525390625, 9076.400390625, 9562.6826171875, 10065.978515625, 10584.6318359375, 11116.662109375, 11660.0673828125, 12211.5478515625, 12766.873046875, 13324.6689453125, 13881.3310546875, 14432.1396484375, 14975.615234375, 15508.2568359375, 16026.115234375, 16527.322265625, 17008.7890625, 17467.61328125, 17901.62109375, 18308.43359375, 18685.71875, 19031.2890625, 19343.51171875, 19620.04296875, 19859.390625, 20059.931640625, 20219.6640625, 20337.86328125, 20412.30859375, 20442.078125, 20425.71875, 20361.81640625, 20249.51171875, 20087.0859375, 19874.025390625, 19608.572265625, 19290.2265625, 18917.4609375, 18489.70703125, 18006.92578125, 17471.83984375, 16888.6875, 16262.046875, 15596.6953125, 14898.453125, 14173.32421875, 13427.76953125, 12668.2578125, 11901.33984375, 11133.3046875, 10370.17578125, 9617.515625, 8880.453125, 8163.375, 7470.34375, 6804.421875, 6168.53125, 5564.3828125, 4993.796875, 4457.375, 3955.9609375, 3489.234375, 3057.265625, 2659.140625, 2294.2421875, 1961.5, 1659.4765625, 1387.546875, 1143.25, 926.5078125, 734.9921875, 568.0625, 424.4140625, 302.4765625, 202.484375, 122.1015625, 62.78125, 22.8359375, 3.75781297683716, 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 3.81999996079685e-08, 6.76070021654596e-06, 2.43480008066399e-05, 5.89219998801127e-05, 0.000111914298031479, 0.00019857739971485, 0.000340379687258974, 0.000561555323656648, 0.000889697927050292, 0.00135280552785844, 0.00199183798395097, 0.00285712420009077, 0.00397095363587141, 0.00537781463935971, 0.00713337678462267, 0.00926146004348993, 0.0118060223758221, 0.014815628528595, 0.0183184519410133, 0.0223548449575901, 0.0269635207951069, 0.0321760959923267, 0.03802639991045, 0.0445479601621628, 0.051773015409708, 0.0597284138202667, 0.0684482529759407, 0.077958308160305, 0.0882857367396355, 0.099461667239666, 0.111504651606083, 0.124448128044605, 0.138312891125679, 0.153125032782555, 0.168910413980484, 0.185689449310303, 0.2034912109375, 0.222332864999771, 0.242244005203247, 0.26324188709259, 0.285354018211365, 0.308598458766937, 0.332939088344574, 0.358254194259644, 0.384363323450089, 0.411124765872955, 0.438391208648682, 0.46600329875946, 0.493800312280655, 0.521619200706482, 0.549301147460938, 0.576692163944244, 0.603648066520691, 0.630035817623138, 0.655735969543457, 0.680643022060394, 0.704668998718262, 0.727738738059998, 0.749796569347382, 0.770797550678253, 0.790716767311096, 0.809536039829254, 0.827256083488464, 0.843881130218506, 0.859431803226471, 0.873929262161255, 0.887407541275024, 0.899900496006012, 0.911448180675507, 0.922095656394958, 0.9318807721138, 0.94085955619812, 0.949064433574677, 0.956549525260925, 0.963351726531982, 0.969513416290283, 0.975078403949738, 0.980071604251862, 0.984541893005371, 0.988499522209167, 0.991984009742737, 0.995002508163452, 0.99763011932373, 1. ;
string aermr04:GRIB_shortName = "aermr04" ;
string aermr04:GRIB_units = "kg kg**-1" ;
string aermr04:long_name = "Dust Aerosol (0.03 - 0.55 um) Mixing Ratio" ;
string aermr04:units = "kg kg**-1" ;
string aermr04:standard_name = "unknown" ;
string aermr04:coordinates = "step time hybrid latitude longitude valid_time" ;
// global attributes:
string :GRIB_centre = "ecmf" ;
string :GRIB_centreDescription = "European Centre for Medium-Range Weather Forecasts" ;
:GRIB_subCentre = 0LL ;
string :Conventions = "CF-1.7" ;
string :institution = "European Centre for Medium-Range Weather Forecasts" ;
string :history = "2024-09-06T16:46 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {\"source\": \"data.grib\", \"filter_by_keys\": {\"typeOfLevel\": \"hybrid\"}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
CAMS global reanalysis (EAC4)
Legacy converter
netcdf camseac4old {
dimensions:
longitude = 480 ;
latitude = 241 ;
level = 2 ;
time = 1 ;
variables:
float longitude(longitude) ;
longitude:units = "degrees_east" ;
longitude:long_name = "longitude" ;
float latitude(latitude) ;
latitude:units = "degrees_north" ;
latitude:long_name = "latitude" ;
int level(level) ;
level:units = "millibars" ;
level:long_name = "pressure_level" ;
int time(time) ;
time:units = "hours since 1900-01-01 00:00:00.0" ;
time:long_name = "time" ;
time:calendar = "gregorian" ;
short c2h6(time, level, latitude, longitude) ;
c2h6:scale_factor = 1.6469724757173e-17 ;
c2h6:add_offset = 5.52540709711356e-13 ;
c2h6:_FillValue = -32767s ;
c2h6:missing_value = -32767s ;
c2h6:units = "kg kg**-1" ;
c2h6:long_name = "Ethane" ;
c2h6:standard_name = "mass_fraction_of_ethane_in_air" ;
// global attributes:
:Conventions = "CF-1.6" ;
:history = "2024-09-06 16:46:49 GMT by grib_to_netcdf-2.25.1: /opt/ecmwf/mars-client/bin/grib_to_netcdf.bin -S param -o /cache/data6/adaptor.mars.internal-1725641209.203058-26900-7-448cb5be-9939-4e25-a58a-1d06dd2c856f.nc /cache/tmp/448cb5be-9939-4e25-a58a-1d06dd2c856f-adaptor.mars.internal-1725641208.8677917-26900-12-tmp.grib" ;
} |
Main differences
|
New converter
netcdf camseac4new {
dimensions:
valid_time = 1 ;
pressure_level = 2 ;
latitude = 241 ;
longitude = 480 ;
variables:
int64 valid_time(valid_time) ;
string valid_time:long_name = "time" ;
string valid_time:standard_name = "time" ;
string valid_time:units = "seconds since 1970-01-01" ;
string valid_time:calendar = "proleptic_gregorian" ;
double pressure_level(pressure_level) ;
pressure_level:_FillValue = NaN ;
string pressure_level:long_name = "pressure" ;
string pressure_level:units = "hPa" ;
string pressure_level:positive = "down" ;
string pressure_level:stored_direction = "decreasing" ;
string pressure_level:standard_name = "air_pressure" ;
double latitude(latitude) ;
latitude:_FillValue = NaN ;
string latitude:units = "degrees_north" ;
string latitude:standard_name = "latitude" ;
string latitude:long_name = "latitude" ;
string latitude:stored_direction = "decreasing" ;
double longitude(longitude) ;
longitude:_FillValue = NaN ;
string longitude:units = "degrees_east" ;
string longitude:standard_name = "longitude" ;
string longitude:long_name = "longitude" ;
float c2h6(valid_time, pressure_level, latitude, longitude) ;
c2h6:_FillValue = NaNf ;
c2h6:GRIB_paramId = 217045LL ;
string c2h6:GRIB_dataType = "an" ;
c2h6:GRIB_numberOfPoints = 115680LL ;
string c2h6:GRIB_typeOfLevel = "isobaricInhPa" ;
c2h6:GRIB_stepUnits = 1LL ;
string c2h6:GRIB_stepType = "instant" ;
string c2h6:GRIB_gridType = "regular_ll" ;
c2h6:GRIB_uvRelativeToGrid = 0LL ;
c2h6:GRIB_NV = 0LL ;
c2h6:GRIB_Nx = 480LL ;
c2h6:GRIB_Ny = 241LL ;
string c2h6:GRIB_cfName = "mass_fraction_of_ethane_in_air" ;
string c2h6:GRIB_cfVarName = "c2h6" ;
string c2h6:GRIB_gridDefinitionDescription = "Latitude/Longitude Grid" ;
c2h6:GRIB_iDirectionIncrementInDegrees = 0.75 ;
c2h6:GRIB_iScansNegatively = 0LL ;
c2h6:GRIB_jDirectionIncrementInDegrees = 0.75 ;
c2h6:GRIB_jPointsAreConsecutive = 0LL ;
c2h6:GRIB_jScansPositively = 0LL ;
c2h6:GRIB_latitudeOfFirstGridPointInDegrees = 90. ;
c2h6:GRIB_latitudeOfLastGridPointInDegrees = -90. ;
c2h6:GRIB_longitudeOfFirstGridPointInDegrees = 0. ;
c2h6:GRIB_longitudeOfLastGridPointInDegrees = 359.25 ;
c2h6:GRIB_missingValue = 3.40282346638529e+38 ;
string c2h6:GRIB_name = "Ethane" ;
string c2h6:GRIB_shortName = "c2h6" ;
c2h6:GRIB_totalNumber = 0LL ;
string c2h6:GRIB_units = "kg kg**-1" ;
string c2h6:long_name = "Ethane" ;
string c2h6:units = "kg kg**-1" ;
string c2h6:standard_name = "mass_fraction_of_ethane_in_air" ;
c2h6:GRIB_number = 0LL ;
string c2h6:coordinates = "valid_time isobaricInhPa latitude longitude" ;
// global attributes:
string :GRIB_centre = "ecmf" ;
string :GRIB_centreDescription = "European Centre for Medium-Range Weather Forecasts" ;
:GRIB_subCentre = 0LL ;
string :Conventions = "CF-1.7" ;
string :institution = "European Centre for Medium-Range Weather Forecasts" ;
string :history = "2024-09-06T16:48 GRIB to CDM+CF via cfgrib-0.9.14.0/ecCodes-2.36.0 with {\"source\": \"data.grib\", \"filter_by_keys\": {\"typeOfLevel\": \"isobaricInhPa\"}, \"encode_cf\": [\"parameter\", \"time\", \"geography\", \"vertical\"]}" ;
} |
New Coordinate variable names
Some of the coordinate variables have been rename to address issues in the legacy netCDF conversion regarding non CF complaint variables, and unclear and overlapping definitions of the time-dimensions.
Table 1: New Coordinate variable names
| Coordinate variable name in new netCDF conversion | Coordinate variable name in legacy netCDF conversion for regional reanalysis (CARRA/CERRA/UERRA) | Coordinate variable name in legacy netCDF conversion for global reanalysis (ERA5 family), seasonal forecasts and CAMS datasets | GRIB/MARS key(s) | Any other business |
|---|---|---|---|---|
| latitude | latitude | latitude | latitude | |
| longitude | longitude | longitude | longitude | |
| valid_time | valid_time | time | validityDate+ validityTime |
|
| forecast_reference_time | time | time | time | A time dimension for "forecast" data. |
| forecast_period | step | time | step | A time dimension for "forecast" data. |
| indexing_time | indexing_time | time | indexingDate + indexingTime | This coordinate is only used in 'seasonal-monthly-*-levels' and 'seasonal-postprocessed-*' datasets. This accounts for data where the forecast_reference_time differs for each ensemble member, specifically this happens when the ensemble members are initialised following a lagged start approach. In those situations instead of forecast_reference_time a "nominal start date" is used encoded with this name of "indexing_time". |
| forecastMonth | forecastMonth | time | forecastMonth | This coordinate is only used in 'seasonal-monthly-*-levels' and 'seasonal-postprocessed-*' datasets. This is the number of step w.r.t the forecast_reference_time or indexing_time. The convention used for the numbering is "1" is the first complete month after the nominal start date (forecast_reference_time/indexing_time), for instance if the nominal start date is 1st November, forecastMonth=1 means November. |
valid_month (this may be renamed valid_time in future editions) | valid_month | time | monthlyVerificationDate + validityTime | This is required by some monthly datasets where the "valid_time" differs between start of month and end of month, depending on variable. valid_month not used for ERA5 family and CAMS datasets. |
| pressure_level | isobaricInhPa | level | levelist + levtype or level + typeOfLevel | |
| model_level | hybrid | level | levelist + levtype or level + typeOfLevel | |
| number | number | number | number |
Legacy converter
To assist in transition, the legacy convertor will be made available programatically via the cdsapi, however this is considered a deprecated format and is no longer supported. To use the legacy convertor, you should update you cdsapi request with: Please be aware that the resulting files are netCDF3. |
This document has been produced in the context of the Copernicus Atmosphere Monitoring Service (CAMS) and Copernicus Climate Change Service (C3S). The activities leading to these results have been contracted by the European Centre for Medium-Range Weather Forecasts, operator of CAMS and C3S on behalf of the European Union (Delegation Agreement signed on 11/11/2014 and Contribution Agreement signed on 22/07/2021). All information in this document is provided "as is" and no guarantee or warranty is given that the information is fit for any particular purpose. The users thereof use the information at their sole risk and liability. For the avoidance of all doubt , the European Commission and the European Centre for Medium - Range Weather Forecasts have no liability in respect of this document, which is merely representing the author's view. |