ECMWF updated the default versions of its software packages and libraries used on Atos HPCF and ECS as part of the regular update cycle.

Table of contents

Highlights

Python 3.12 becomes default

For our Python users, we now offer a more recent version of Python 3.12 with the usual extra modules found on the previous default 3.11. It features new versions of many extras, like NumPy version 2 and the latest earthkit stack.

Note that new versions of other packages with python bindings may only come with support for the new Python 3.12.

You can check all the extra packages included in this installation with:

module help python3/may25

ECMWF software improvements

There have been a number of improvements and changes in our ECMWF software and libraries since our last default change in November 2024. See the blog posts for all the software releases since then to learn about what is new in ecCodes, Magics or Metview.

ecCodes changes encoding for certain GRIB2 parameters and other changes

Since version 2.28.0, ecCodes has changed the way it encodes certain GRIB2 parameters, in preparation for our Migration to GRIB edition 2. The complete list can be found in Migration to GRIB2 - new in ecCodes . If you encode those particular GRIB2 fields with a more recent ecCodes version, you will not be able to decode them properly with an older ecCodes. We strongly recommend to make sure all your production chain uses an up-to-date ecCodes version for best results.

There has been changes in parameters representing statistical processing in time of an existing parameter with a new structure to streamline and unify statistically processed parameters. 

Additionally, from version 2.40.0, encoding a constant field now sets the binary scale factor to zero and 

• The following C API functions have been removed:
  • codes_context_set_memory_proc
  • codes_context_set_persistent_memory_proc
  • codes_context_set_buffer_memory_proc
• GRIB: Encoding a constant field now sets the binary scale factor to zero. See ECC-2012
• The following C API functions are deprecated and will be removed in a future release:

  codes_string_list* codes_grib_util_get_param_id(const char* mars_param); codes_string_list* codes_grib_util_get_mars_param(const char* param_id);

• The CMake option ENABLE_TIMER is deprecated and will be removed in the next release.
• bufr_dump will fail if an unknown descriptor is encountered.

See ecCodes Latest news for more information.

A lot of other bugfixes and improvements

All the new versions, both ECMWF and 3rd Party, come with a number of bug fixes and improvements. You may see the change logs and news for any individual package or library in the list below if you want to have all the details.

How to use the new versions for testing BEFORE the update

The new versions of the software packages can be selected by using the may25 tag in modules. For example:

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

Change list

ECMWF Software

The new version of the ECMWF Toolbox (ecmwf-toolbox module) is 2025.04.0.0, replacing 2024.09.0.0. This table summarises the changes in the version of the different packages within the bundle:

Besides ecmwf-toolbox, the following ECMWF packages with their corresponding module also see their default changed:

Please see the Release notes for full details of the changes implemented in each package.

Third-party software

Some of the versions in this list may already be the default for a specific package, so no changes will be made in those cases.

Potential issues

ecFlow and python3 incompatibilities

While the ecFlow client and python3 default versions are updated on the HPCF, if you may be running an older version of the ecFlow server. If in your ecFlow tasks you use the ecFlow Python bindings, they may not work with the new default version of Python.

We will update the version of all servers shortly, but in the meantime you may need to either revert to using python3/old in your tasks, or force them to load the new default ecFlow module.

Basemap not present on Python3 3.12

Current versions of Basemap cannot be provided as part of the same environment since they have not been ported to NumPy 2, which is the version used in Python 3.12. If you rely on Basemap, you should stay on Python 3.11 until Basemap has been ported, or move to similar packages such as CartoPy.

Metview and Pyproj / cartopy incompatibilities

When using Metview from Python 3.12, there is a known incompatibility with any package that uses Pyproj, such as Cartopy. If Metview is imported first, then importing the incompatible packages may fail with:

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

Avoiding the use of both in the same script is the safest approach. However, if that is not possible, switching the order of imports so metview is imported last may solve the issue. For example:

import cartopy
import metview

This issue has been addressed and should no longer be occurring.