We are in the process of updating the Operating System on all complexes of our Atos HPCF, from RedHat RHEL 8.6 to RHEL 8.8.
The default Member-State user complex AC will be updated on:
11 June 2024 from 08:00 UTC
All jobs launched after that time, including interactive ones, will dispatch to nodes already updated to RHEL 8.8. New logins via "ssh hpc-login" from 08:00 onwards will also land on updated nodes.
Two complexes, AB and AD, have already been through the update, as well as the GPU partition on AC. The last complex AA will follow to be upgraded to 8.8 a few days after AC.
The update itself should be completely harmless and transparent in most cases, except for those using the vendor provided OpenMPI, typically loaded via module load openmpi. On the updated OS image the default version of vendor-provided, vendor-supported OpenMPI changes from 4.1.1.1 to 4.1.5.4.
To facilitate the transition, we have backported OpenMPI 4.1.1.1 and basic tests with it on the new OS have been successful. However, since it is not officially supported, you may use it at your own responsibility, and it may not be possible to fix any problems you may encounter; even so, if you find any issues trying to run with OpenMPI 4.1.1.1, for our awareness please let us know by reporting this through https://support.ecmwf.int/
We strongly recommend to switch to fully supported alternatives with either module load hpcx-openmpi, the MPI version used in the operational production at ECMWF, or the vendor-provided and supported version referenced by module load openmpi.
You may test your usual workloads on AD before the update on AC to ensure everything works as expected:
- Login to ad-login instead of hpc-login and submit your tests jobs from there
- If submitting jobs remotely, use ad-batch instead of hpc-batch.
- If using troika for remote submission of tasks from ecFlow, temporarily replace "hpc" by "ad" as your target.
Once the tests are done, please restore your target host to the original value.
Please be aware that the small GPU partition in complex AC has already been upgraded to rhel 8.8.
You may get in touch with us for any concerns of doubts by raising an issue through our ECMWF Support Portal.