ECMWF updated the default versions of its software packages and libraries used on Atos HPCF and ECS as part of the regular update cycle.
When?
The changes took place on Wednesday 15 May 2024 09:00 UTC
Do I need to do anything?
We strongly recommend you test that your programs, scripts and suites work with the new versions of any software or library affected before the change happens.
Table of contents
Highlights
Python 3.11 becomes default
For our Python users, we now offer a more recent version of Python 3.11 with the usual extra modules found on the previous default 3.10. Note that new versions of other packages with python bindings may only come with support for the new Python 3.11.
Enhanced experience for the new ECMWF Jupyterhub service
We have recently opened our new ECMWF Jupyterhub service at https://jupyterhub.ecmwf.int. to all users of our HPCF and ECS. While the old software stack is also supported, you will benefit from an updated Jupyter environment and tools. You can find more information about this new way of working on the HPCF and ECS in the ECMWF JupyterHub documentation pages.
ECMWF software improvements
There have been a vast number of improvements and changes in our ECMWF software and libraries since our last default change in November 2023. See the blog posts for all the software releases since then to learn about what is new in ecCodes, Magics or Metview.
ecCodes changes encoding for certain GRIB2 parameters and other changes
Since version 2.28.0, ecCodes has changed the way it encodes certain GRIB2 parameters, in preparation for our Migration to GRIB edition 2. The complete list can be found in Migration to GRIB2 - new in ecCodes . If you encode those particular GRIB2 fields with a more recent ecCodes version, you will not be able to decode them properly with an older ecCodes. We strongly recommend to make sure all your production chain uses an up-to-date ecCodes version for best results.
Additionally, the key "levtype
" is now read-only (for GRIB edition 2). You should use "typeOfLevel
" to set fixed surface types.
For consistency, the key "validityTime" now has 4 characters as a string like dataTime. This affects the output of "grib_ls -n time
"
A lot of other bugfixes and improvements
All the new versions, both ECMWF and 3rd Party, come with a number of bug fixes and improvements. You may see the change logs and news for any individual package or library in the list below if you want to have all the details.
How to use the new versions for testing BEFORE the update
Users are strongly encouraged to test their software applications and data processing chain with the new versions of the various software packages before the date of the change to the default versions.
The new versions of the software packages can be selected by using the may24 tag in modules. For example:
|
How to revert to the old versions in case of problems AFTER the update
Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:
|
Please let ECMWF know if you need to revert to the old version of any of the packages after the update of the defaults has taken place.
Change list
ECMWF Software
The new version of the ECMWF Toolbox (ecmwf-toolbox
module) is 2024.04.0.0, replacing 2023.10.1.0. This table summarises the changes in the version of the different packages within the bundle:
Package | Default version | Release notes | Download page | |
---|---|---|---|---|
Before | After | |||
CodesUI | 1.7.4 | 1.8.0 | CodesUI - Change History | CodesUI - Releases |
ecCodes | 2.32.1 | 2.35.0 | ecCodes Releases | |
Magics | 4.14.1 | 4.15.4 | Latest News | Magics Releases |
Metview | 5.20.0 | 5.22.0 | Metview Releases | |
ODC | 1.4.6 | 1.5.0 | Readme | ODC Releases |
Besides ecmwf-toolbox
, the following ECMWF packages with their corresponding module also see their default changed:
ecFlow | 5.11.4 | 5.12.4 | ecFlow Releases | |
---|---|---|---|---|
climetlab | 0.12.8 | 0.21.3 | https://github.com/ecmwf/climetlab | https://github.com/ecmwf/climetlab/releases |
Please see the Release notes for full details of the changes implemented in each package.
Third-party software
Some of the versions in this list may already be the default for a specific package, so no changes will be made in those cases.
Potential issues
ecFlow and python3 incompatibilities
While the ecFlow client and python3 default versions are updated on the HPCF, if you may be running an older version of the ecFlow server. If in your ecFlow tasks you use the ecFlow Python bindings, they may not work with the new default version of Python.
We will update the version of all servers shortly, but in the meantime you may need to either revert to using python3/old in your tasks, or force them to load the new default ecFlow module.