ECMWF updated the default versions of its software packages and libraries used on Atos HPCF and ECS as part of the regular update cycle.

Table of contents

Highlights

Python 3.11 becomes default

For our Python users, we now offer a more recent version of Python 3.11 with the usual extra modules found on the previous default 3.10. Note that new versions of other packages with python bindings may only come with support for the new Python 3.11.

Enhanced experience for the new ECMWF Jupyterhub service

We have recently opened our new ECMWF Jupyterhub service at https://jupyterhub.ecmwf.int. to all users of our HPCF and ECS.  While the old software stack is also supported, you will benefit from an updated Jupyter environment and tools. You can find more information about this new way of working on the HPCF and ECS in the ECMWF JupyterHub documentation pages.

ECMWF software improvements

There have been a vast number of improvements and changes in our ECMWF software and libraries since our last default change in November 2023. See the blog posts for all the software releases since then to learn about what is new in ecCodes, Magics or Metview.

• December 2023 software release
• April 2024 software release

ecCodes changes encoding for certain GRIB2 parameters and other changes

Since version 2.28.0, ecCodes has changed the way it encodes certain GRIB2 parameters, in preparation for our Migration to GRIB edition 2. The complete list can be found in Migration to GRIB2 - new in ecCodes . If you encode those particular GRIB2 fields with a more recent ecCodes version, you will not be able to decode them properly with an older ecCodes. We strongly recommend to make sure all your production chain uses an up-to-date ecCodes version for best results.

Additionally, the key "levtype" is now read-only (for GRIB edition 2). You should use "typeOfLevel" to set fixed surface types. 

For consistency, the key "validityTime" now has 4 characters as a string like dataTime. This affects the output of "grib_ls -n time"

A lot of other bugfixes and improvements

All the new versions, both ECMWF and 3rd Party, come with a number of bug fixes and improvements. You may see the change logs and news for any individual package or library in the list below if you want to have all the details.

How to use the new versions for testing BEFORE the update

The new versions of the software packages can be selected by using the may24 tag in modules. For example:

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

Change list

ECMWF Software

The new version of the ECMWF Toolbox (ecmwf-toolbox module) is 2024.04.0.0, replacing 2023.10.1.0. This table summarises the changes in the version of the different packages within the bundle:

Besides ecmwf-toolbox, the following ECMWF packages with their corresponding module also see their default changed:

Please see the Release notes for full details of the changes implemented in each package.

Third-party software

Some of the versions in this list may already be the default for a specific package, so no changes will be made in those cases.

Potential issues

ecFlow and python3 incompatibilities

While the ecFlow client and python3 default versions are updated on the HPCF, if you may be running an older version of the ecFlow server. If in your ecFlow tasks you use the ecFlow Python bindings, they may not work with the new default version of Python.

We will update the version of all servers shortly, but in the meantime you may need to either revert to using python3/old in your tasks, or force them to load the new default ecFlow module.