ECMWF is updating the default versions of its software packages and libraries used across all user platforms at the centre as part of the regular update cycle.
When?
The changes will take place on Wednesday 25 November 2020 10:00 - 10:30 UTC
What platforms are affected?
All user platforms at the centre will see their default additional software stack updated:
- All clusters in our High Performance Computing Facility, mainly CCA and CCB
- ECGATE
- Linux Clusters
- Linux Desktops
The next defaults are tagged as "nov20" in modules. After the session the previous defaults will be tagged as "old"
Table of contents
Highlights
A number of bug fixes and performance improvements have been introduced on the different versions to become default. Here are some of the most relevant new features in respect to the previous default set of versions:
- ecCodes, the heart of GRIB and BUFR handling, brings performance improvements, new parameters, contributions from users and several smaller features/bug fixes including support for the Mercator projection. Please note that ecCodes' Python 2 interface is no longer being developed. Users are strongly advised to migrate to Python 3.
- Magics, ECMWF's plotting library, adds some smaller fixes and additional improvements including :
- combined with the latest ecCodes, Magics brings plotting support, including interactive Cursor Data, for GRIBs encoded with various projections such as Mercator and Lambert variants.
- a major code clean-up has reduced the number of dependencies. The Boost library and Python are not required to install Magics anymore.
- fix issue with logging that was degrading observation plotting speed when used from Python
- fix issue in python when passing single element array.
- combined with the latest ecCodes, Magics brings plotting support, including interactive Cursor Data, for GRIBs encoded with various projections such as Mercator and Lambert variants.
- Metview, ECMWF's interactive and batch software for manipulation and plotting of data, brings several new features and functions, such as:
- much improved speed of plotting in-memory arrays of data using the Input Visualiser thanks to allowing vectors (Macro language) or numpy arrays (Python) to be used as data
- the last two modules that were originally implemented in Fortran (Spectra and Potential Temperature) have been converted to C++, meaning they will be available to more users
- fixes for macOS
- new Gallery examples
- a new regridding module that exposes many new interpolation methods for expert users
- user interface improvements such as a new preview panel that allows quick inspection of icons
- the long-awaited ability to filter for parameters in the icon editors makes its way into this release
- a whole raft of new Python/Macro functions for thermodynamic computations and plotting utilities
- more - see the release notes!
Metview's new Regrid module brings new possibilities in gridding and transformation of data
- CodesUI, the standalone combination of Metview's GRIB and BUFR examiners, added syntax highlighting for textual GRIB dumps and fixed an issue where the BUFR filter could unexpectedly crash.
How to use the new versions for testing BEFORE the update
The new versions of the software packages can be selected by using the nov20 tag in modules. For example:
module swap metview/nov20 module load Magics/nov20 module load bufrdc/nov20 module swap eccodes/nov20
How to revert to the old versions in case of problems AFTER the update
Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:
module swap metview/old module load Magics/old module load bufrdc/old module swap eccodes/old
Version changes
This table summarises the changes in the version of the different packages:
Package | Default version | Release notes | Download page | |
---|---|---|---|---|
Before | After | |||
CodesUI | 1.3.1 | 1.4.1 | CodesUI - Releases | |
ecCodes | 2.17.1 | 2.19.1 | Latest news | ecCodes Releases |
ecFlow | 4.17.2 | 4.17.2 (no change) | ecFlow Releases | |
ecFlow/5 | 5.4.0 | 5.5.3 | ecFlow Releases | |
Magics | 4.3.3 | 4.5.1 | Latest News | Magics Releases |
Metview | 5.8.3 | 5.10.1 | Metview Releases Metview Bundle | |
ODB API | 0.19.4 | 0.19.5 | Change history | ODB API Releases |
ODC | 1.0.0 | 1.1.1 | - | - |
Please see the Release notes for full details of the changes implemented in each package.
ecCodes is now the official ECMWF encoder and decoder for GRIB and BUFR formats. It superseded GRIB-API, for the GRIB format, and EMOSLIB, for the BUFR. If you still use any of those, we encourage you to migrate to ecCodes. EMOSLIB is, in fact, no longer updated. The standalone BUFRDC package is also installed to help in the migration.
Package | Default version | |
---|---|---|
Before | After | |
BUFRDC | 4.6.3 | 4.6.4 |
Caveats
ecCodes 2.19 and Snow Depth Parameter (141)
ParameterNumber for Snow Depth (141) is now 254 instead of 11 when encoding in GRIB2 format.
The GRIB edition 2 for 141 encoding was wrong in previous versions of ecCodes: The parameterNumber should be 60, not 11 because 11 is the physical depth not the one converted to water volume.
In older versions, both paramIds 141 and 3066 shared the same GRIB2 representation and the consequence of that was 141 was obfuscated by 3066 (water converted depth is obfuscated but the true physical depth!)
From ecCodes 2.19 onwards, this parameter uses local GRIB2 encoding in ECMWF local table 1:
discipline = 0; parameterCategory = 1; parameterNumber = 254; localTablesVersion = 1 ;