ECMWF is updating the default versions of its software packages and libraries used on Atos HPCF and ECS as part of the regular update cycle.

Table of contents

Highlights

Python 3.11 becomes default

For our Python users, we now offer a more recent version of Python 3.11 with the usual extra modules found on the previous default 3.10. Note that new versions of other packages with python bindings may only come with support for the new Python 3.11.

Enhanced experience for the new ECMWF Jupyterhub service

We have recently opened our new ECMWF Jupyterhub service at https://jupyterhub.ecmwf.int. to all users of our HPCF and ECS.  While the old software stack is also supported, you will benefit from an updated Jupyter environment and tools. You can find more information about this new way of working on the HPCF and ECS in the ECMWF JupyterHub documentation pages.

ECMWF software improvements

There have been a vast number of improvements and changes in our ECMWF software and libraries since our last default change in November 2023. See the blog posts for all the software releases since then to learn about what is new in ecCodes, Magics or Metview.

• December 2023 software release
• April 2024 software release

ecCodes changes encoding for certain GRIB2 parameters and other changes

Since version 2.28.0, ecCodes has changed the way it encodes certain GRIB2 parameters, in preparation for our Migration to GRIB edition 2. The complete list can be found in Migration to GRIB2 - new in ecCodes . If you encode those particular GRIB2 fields with a more recent ecCodes version, you will not be able to decode them properly with an older ecCodes. We strongly recommend to make sure all your production chain uses an up-to-date ecCodes version for best results.

Additionally, the key "levtype" is now read-only (for GRIB edition 2). You should use "typeOfLevel" to set fixed surface types. 

For consistency, the key "validityTime" now has 4 characters as a string like dataTime. This affects the output of "grib_ls -n time"

A lot of other bugfixes and improvements

All the new versions, both ECMWF and 3rd Party, come with a number of bug fixes and improvements. You may see the change logs and news for any individual package or library in the list below if you want to have all the details.

How to use the new versions for testing BEFORE the update

The new versions of the software packages can be selected by using the may24 tag in modules. For example:

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

Change list

ECMWF Software

The new version of the ECMWF Toolbox (ecmwf-toolbox module) is 2024.04.0.0, replacing 2023.10.1.0. This table summarises the changes in the version of the different packages within the bundle:

Besides ecmwf-toolbox, the following ECMWF packages with their corresponding module also see their default changed:

Please see the Release notes for full details of the changes implemented in each package.

Third-party software

Some of the versions in this list may already be the default for a specific package, so no changes will be made in those cases.

We are thrilled to announce the general availability of the new JupyterHub service at ECMWF, a new way of accessing computing resources at ECMWF in an interactive and modern way. With JupyterHub you can now spin up your JupyterLab sessions on multiple backends including HPCF and ECS, leveraging the computational resources available at ECMWF without leaving your browser.

For those unfamiliar, Jupyter is an open-source platform that provides an interactive computing environment where users can manipulate data, run code, and visualise results in real-time, fostering a seamless and efficient workflow. It enables users to create and share documents that contain live code, equations, visualisations, and narrative text. It supports various programming languages, including Python, R, and Julia, making it a versatile tool for data analysis, machine learning, scientific computing, and more.

Jupyter notebooks capture the entire computational process, from data ingestion to analysis and visualisation, ensuring reproducibility of results. This feature is invaluable for research, as it allows others to reproduce and validate findings easily. Jupyter also offers rich visualisation capabilities, allowing users to create interactive plots, charts, and graphs directly within their notebooks.

You can find more information about the service and how to use it in our JupyterHub documentation page.

ECMWF is updating the default versions of its software packages and libraries used across on Atos HPCF and ECS as part of the regular update cycle.

Table of contents

Highlights

There have been a number of bug fixes and improvements on our ECMWF software and libraries since our last default change in May 2023. See the blog posts for all the software releases since then to learn about what is new in ecCodes, Magics or Metview.

• July 2023 software release
• October 2023 software release

How to use the new versions for testing BEFORE the update

The new versions of the software packages can be selected by using the nov23 tag in modules. For example:

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

Change list

The new version of the ECMWF Toolbox (ecmwf-toolbox module) is 2023.10.1.0, replacing 2023.04.1.0. This table summarises the changes in the version of the different packages within the bundle:

Besides ecmwf-toolbox, the following ECMWF packages with their corresponding module also see their default changed:

Please see the Release notes for full details of the changes implemented in each package.

ECMWF is updating the default versions of its software packages and libraries used across on Atos HPCF and ECS as part of the regular update cycle.

Table of contents

Highlights

Software stack ready for next operational IFS cycle

With the upcoming Implementation of IFS Cycle 48r1, this set of defaults is ready to handle the data produced.

Python 3.10 becomes default

For our Python users, we offer now a more recent version of Python 3.10 with the usual extra modules found on the previous default 3.8. Note that new versions of other packages with python bindings may only come with support for the new Python 3.10.

ECMWF software improvements

There. have been a vast number of improvements on our ECMWF software and libraries since our last default change in 2021. See the blog posts for all the software releases since then to learn about what is new in ecCodes, Magics or Metview.

• December 2021 software release
• March 2022 software release
• May 2022 software release
• August 2022 software release
• December 2022 software release
• January 2023 software release
• April 2023 software release

ecCodes changes encoding for certain GRIB2 parameters

Since version 2.28.0, ecCodes has changed the way it encodes certain GRIB2 parameters, in preparation for our Migration to GRIB edition 2. The complete list can be found in Migration to GRIB2 - new in ecCodes . If you encode those particular GRIB2 fields with ecCodes 2.28 or newer, you will not be able to decode them properly with an older ecCodes. We strongly recommend to make sure all your production chain uses an up-to-date ecCodes version for better results.

A lot of other bugfixes and improvements

All the new versions, both ECMWF and 3rd Party, come with a number of bug fixes and improvements, since it has been 2 years since our last change of default software. You may see the Change logs for any individual package or library in the list below if you want to have all the details.

How to use the new versions for testing BEFORE the update

The new versions of the software packages can be selected by using the may23 tag in modules. For example:

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

Change list

ECMWF Software

The new version of the ECMWF Toolbox (ecmwf-toolbox module) is 2023.04.1.0, replacing 2021.08.3.0. This table summarises the changes in the version of the different packages within the bundle:

Besides ecmwf-toolbox, the following ECMWF packages with their corresponding module also see their default changed:

Please see the Release notes for full details of the changes implemented in each package.

Third-party software

Some of the versions in this list may already be the default for a specific package, so no changes will be made in those cases.

Dear ecCodes users,

We wanted to make you aware of these two important changes in the upcoming 2.29.0 release of ecCodes.

First, the old "python2" bindings of ecCodes, those installed directly with the main distribution, will not be available anymore. Python2 has been deprecated since 2020 (see https://www.python.org/doc/sunset-python-2/) and in order to simplify the software we have decided to remove it from the distribution. You can install the Python3 compatible bindings, which are still API compatible with the Python2 ones (that is, you will not have to change your programs beyond adapting them to use Python3) following these instructions ecCodes installation#Python3bindings. In addition to the "pip3" installation, the source code is available in a separate git repository that you can check here https://github.com/ecmwf/eccodes-python, and is also available as a "brew" or "conda" package and as native packages (rpm, deb) for many Linux distributions too.

See also Python 3 interface for ecCodes for further information on the Python3 interface.

Second, from this version ecCodes will be built with a C++ compiler instead of a C compiler, so in case you choose to build from the source code, you will need to make sure that such a compiler is available in your build platform. ecCodes is being built with C++ as a preparation for further evolution in the code, that will take advantage of better object-oriented features, the standard C++ library, and other C++ ECMWF packages.

Even if the software is compiled with C++, the C and Fortran APIs are unchanged, so your programs should be able to link to it without any change. 

Please open a ticket through our support portal (https://support.ecmwf.int) if you have any question or concern.

Kind regards

ECMWF Software Development Team

The ActivIdentity (HID) security token has been used for over a decade at ECMWF, replacing similar RSA tokens which were used before. These tokens provide a second factor for user authentication alongside a user's password, enabling strong authentication for security sensitive services such as login access to ECMWF's Unix systems and services such as ECPDS.

ECMWF is replacing the ActivIdentity tokens with the use of a Time-Based One-Time Password authenticator application, more accurately known as TOTP Client ("time-based one time password"). Typically, the TOTP clients are implemented on a smartphone, but they can also be installed in your laptop or any other computing devices. The Client is synchronised with systems at ECMWF by following the instructions at TOTP: How to activate. Clients usually use PIN codes, or biometric features, for protection.

In the case of a TOTP App installed on a smartphone, after the initial synchronisation, the One-Time Password ( OTP ) is displayed in the application and automatically changes every 30 seconds.

TOTP Apps have the following beneficial properties:

• Users are free to choose the TOTP App they use, as this is a widely used open standard for second factor authentication. You can find apps for this in the Apple Store, Google Store or Mac Application Store, some of them free, some of them commercial
• Management is self-service for the user through the use of QR Codes (2D barcodes) to establish synchronisation with ECMWF's systems.

What you need to do now

• The current HID tokens (the white small keyboard or phone app) will stop working at the end of March 2023 so please ensure you are migrated by then 
• Do remember that you can use your HID again (until the end of March) if you have any problem with the OTP, so there is no harm in testing the new system. Just delete all your OTP tokens and the HID will be enabled again.
• You may consider installing an TOTP application in your desktop computer in addition to your phone. If you have more than one TOPT you will need to specify which one you are using when logging in.
• ECPDS and ECaccess (boaccess.ecmwf.int) accept only the first TOTP configured. When the system asks for a passcode you should use the first TOTP you have ever configured. This is not indicated in the ECPDS/ECaccess login page.

Main OTP software options

• Google Authenticator
• Microsoft Authenticator
• LastPass Authenticator
• Red Hat FreeOTP

On a laptop

A few users have had good experiences with

• StepTwo (available for Mac and iPhone)
• OTP Manager

Of course, with OTP being a standard, you can find many other apps.

See for more information Using Time-based One-Time Passwords, and in particular TOTP: How to activate and TOTP: How to use

On 29.11.2022 the SBU Accounting app will be migrated to the new Web infrastructure in Bologna and move from the current https://apps.ecmwf.int/sbu-accounting/ to its new URL  

    https://hpc-usage.ecmwf.int

On 1 Nov 2022 we started to charge for Atos HPC usage. At this occasion we would like to encourage all those using or monitoring Atos HPC resources to login to the SBU Accounting app and check if the reported usage on the Atos is in-line with expectations. Please report any unexpected SBU usage via our Support Portal. For more information please also refer to Atos accounting documentation.

With the DHS Move progressing well and the IFS Cycle 47r3 Release Candidate on the Atos on track for implementation on 18 October we would like to give you a further update on the remaining migration steps and any action you might need to take:  

• The Atos system, replacing both the Cray HPCF and ecgate, is now supported 24/7, making it particularly suitable for running time-critical applications. From 1 November usage of the operational Atos HPC service (hpc-login) will be charged against SBU allocations. The Atos ECS service (ecs-login), which replaces ecgate, will remain free of charge.
• After 31 October the Cray HPCF and all attached filesystems will no longer be available. The Cray will be immediately shutdown by the vendor and users must copy all files they need to keep to the Atos before that date.
• From 1 November a full general user service on ecgate will no longer be available. All users are urged to migrate their activities as early as possible and copy all files still needed to the Atos before that date.
• Users of the Time-critical option 1 (TC1) framework should now migrate their activities to the Atos, preferably by 18 October, the day of the operational switch-over. Once migrated please stop running your TC1 jobs in Reading. This framework will still be fully supported on ecgate until 31 October.
• A reduced ecgate service will still be available from 1 November until 13 December, mainly to run TC1 activities. As ECMWF’s focus is on the systems in Bologna, ecgate will only be supported on a least-effort basis.
• Owners of Time-critical option 2 suites should continue testing their workflow on the Atos HPC. We are contacting those owners to discuss the move of their suites to the Atos HPC.
• Users of remote ECaccess gateways are advised to check with their Computing Representative regarding the availability of new gateways, necessary to connect to the ECMWF gateways in Bologna, boaccess.ecmwf.int. On these new gateways users will need to re-create their ectrans associations.
• While the DHS Move is still in progress please watch our detailed Move page for further updates and availability of data.
• To run graphical applications in Bologna, e.g. ecFlow_ui, Metview, we will offer VDI Virtual Linux Desktops in addition to SSH access via Teleport.
• The migration of ECPDS (the ECMWF Production Data Store) to Bologna will take place over a 2-week period, starting from 4 October, see our ECPDS migration page.  

To support users with their migration we are offering Atos expert sessions. Please see the event page for more details and to register.  

The information above is based on current expectations. For more details and to get notified about further updates and changes, please “watch” the relevant migration pages and the Atos documentation.  

Thank you for your patience and continued co-operation during this significant migration to our new Data Centre.

As the transition to our new Data Centre in Bologna is approaching its final phase and we are getting closer to the decommissioning of the data centre in Reading we would like to update you on the remaining migration steps and any action you might need to take:  

• The Atos system, replacing both the Cray HPCF and ecgate, is still awaiting formal acceptance but is providing a full parallel service, reliably supporting the IFS Cycle 47r3 Release Candidate Phase. As we are getting closer to the switch of the operational forecast, the system support will be heightened from the current ‘social hours’ to 24/7. From 1 November usage of the operational Atos HPC service (hpc-login) will be charged against SBU allocations. The Atos ECS service (ecs-login) which replaces ecgate will remain free of charge.
• All users are invited to migrate their workload to the Atos at their earliest convenience.
• From 1 November ecgate, the Cray HPCF, and all attached filesystems will no longer be available. The Cray HPCF will be immediately shutdown by the vendor and the decommissioning of the Reading data centre will start. Please copy all Cray and ecgate files you need to keep to the Atos before that date.
• The DHS Move, expected to cause 5-6 weeks of reduced MARS and ECFS service, will start on 8 September.
• The switch of ECMWF’s operational forecast (Cycle 47r3) to the Atos is scheduled for 18 October and Release Candidate test data has been available since 19 July.
• Time-critical option 1 framework has been set up on the Atos (HPC and ECS) and events are being triggered by the ECMWF operational suite run on the Cray HPCF. Jobs can also be triggered by a smaller number of events linked to the availability of the Release Candidate test data being produced on the Atos HPC. At present, these events can be used for testing.  We will inform users when the time-critical workflow can be moved to the Atos HPC and jobs running on ecgate or the Cray can be stopped.
• Owners of Time-critical option 2 suites should start testing their workflow on the Atos HPC. Operational versions of the suites should continue to run on the Cray until further notice.  We will inform owners when suites can be moved to the Atos HPC.
• The ECaccess Web Toolkit services, such as job submission, file transfers and ectrans have been set up on Atos HPCF with the new ECaccess gateway boaccess.ecmwf.int. Remote ECaccess Web Toolkits should be able to interact with this new gateway. However, we are still working on the interoperability with other ECaccess gateways. Remote sites may need to install new gateway versions.
• To prepare for the migration of ECPDS (the ECMWF Production Data Store) to Bologna all real-time dissemination users are invited to participate in tests to receive Release Candidate test data from Bologna, scheduled for September.

The information above is based on current expectations. For more details and to get notified about further updates and changes, please “watch” the relevant migration pages and the Atos documentation.  

To support users with their migration we are offering Atos expert sessions. Please see the event page for more details and to register.  

Thank you for your patience and continued co-operation during this significant migration to our new Data Centre.

As of 23 March, the Climate Data Store (CDS) and Atmosphere Data Store (ADS) are the first two user facing services operationally supported from our new Data Centre in Bologna. Following that successful migration, we would like to update you on the next steps in the transition to the Bologna Data Centre.

The Atos system, the central component of the new infrastructure in Bologna, was initially expected to start its operational service on 1 May 2022. As additional work is required to ensure that our standards and expectations regarding this system are fully met before it can be accepted, the initial start date for the operational Atos service is unfortunately no longer feasible. This will have the following main implications for the further migration:  

• The operational Atos HPC service will start on 1 September 2022 at the earliest. Until then users will not be charged System Billing Units against their allocation. HPC users are encouraged to continue the testing and migration of their applications to the Atos.
• The switch of ECMWF’s operational forecast (Cycle 47r3) to the Atos is now expected for late September 2022. Initial technical beta test data produced on the Atos is now available. The Release Candidate Phase will start at least 3 months before the operational switch.   
• The migration of real-time dissemination users to ECPDS ( the ECMWF Production Data Store) in Bologna is scheduled to happen in September 2022, close to, or on the day of the operational forecast switch-over.
• The DHS Move, causing 28 days of reduced MARS and ECFS service, is expected to start in mid-June 2022 . Two short dry-run s with reduced MARS and ECFS service for 12-48 hours, are scheduled for 5 April and 26-28 April.
• Users of the ecgate service will be invited later in April 2022 to start migrating their applications to the on the Atos system.
• Both the Cray and ecgate system will be switched off on 1 October 2022 at the earliest.

The information above is based on current expectations. For more details and to get notified about further updates and changes, please “watch” the relevant migration pages.  

Thank you for your patience during this significant migration to our new data centre which will enable exciting scientific developments and an overall better service to all ECMWF users.

Following the termination of the test Atos system (TEMS) in Reading in November 2021 and a lot of work by many teams across ECMWF, we are pleased to announce the availability of the first Atos cluster, named 'aa', in Bologna, for Member State users of our High Performance Facility (HPCF).  

If you are currently using our Cray HPCs (cca and ccb), we invite you to start testing your activities on the first Atos system (aa). To help you with this work, we have made available the Atos HPCF Documentation. We strongly encourage you to read carefully through those pages before you start any tests on this new platform. You have more details on this in the HPC2020: News Feed, and we recommend you watch that for updates on the Atos services in Bologna.

Thank you for reporting any issues or sharing your feedback via the ECMWF Support Portal mentioning 'Atos' in the title.