ECMWF is updating the default versions of its software packages and libraries used across all user platforms at the centre as part of the regular update cycle.
When?
The changes will take place on 5 June 2019 10:00 - 10:30 UTC
What platforms are affected?
All user platforms at the centre will see their default additional software stack updated:
- All clusters in our High Performance Computing Facility, mainly CCA and CCB
- ECGATE
- Linux Clusters
- Linux Desktops
Do I need to do anything?
We strongly recommend you test that your programs, scripts and suites work with the new versions of any software or library affected before the change happens.
In particular, if using ecFlow, please make sure you check the triggers in your suites are well defined as described below.
Table of contents
Highlights
Software stack ready for next operational IFS cycle
With the upcoming Implementation of IFS cycle 46r1, this set of defaults is ready to handle the data produced, including all the new parameters introduced in this cycle.
Default GNU compiler updated
The GNU GCC compiler is used by default in our Linux Clusters. After this session, version 7.3 of this popular compiler will be used by default, instead of the old 6.3.
Python 3 Ready
A more complete Python 3 deployment has been made, with all major ECMWF packages supporting it: Magics, Metview, ecFlow and ecCodes (in beta). We encourage all Python users to start migrating their programs to Python 3, as the official support for Python 2 will cease at the end of 2019. Please have a look at the Migration to Python 3 documentation page for more details on how to do this migration.
Metview new features
The new Metview 5.5 features a new set of thermodynamic functions and lots of new functionality for the Geopoints format. It also uses the new interpolation package MIR by default, replacing the old EMOSLIB interpolation library.
See the Magics and Metview latest news for a more detailed list of changes.
Improved ecFlowUI
This ecFlow release introduces the Timeline View in ecFlowUI. This is available as a tab in the Info Panel and provides a graphical view of node state history taken from the log file, allowing greater overview and analysis of node start and run times. It also supports a Duration view mode, displaying the submission and active times. This mode allows you to filter the nodes of your choice. The Timeline View provides the ideal tool to help you identify and optimise the tasks that are taking too much time.
Update of main third-party software package defaults across all platforms
Once a year we take the opportunity to update not only the software we develop at ECMWF, but also the core 3rd party packages and libraries in our software stack. Amongst the affected packages we can find popular tools and libraries like Python, R, cdo or netCDF 4. See below for the complete list.
Discontinuation of old PGI compilers
Up to now, we have been providing our software libraries built with a legacy PGI 14 compiler for the FORTRAN parts of the code. This helped with the migration from older platforms to new ones where our default compiler of choice has been the GNU GCC. The new versions becoming default now are no longer available for the PGI environment on those platforms where it is present. If you still use it, please consider moving to the default environment. Note that older versions of our software stack will still be available for a bit longer in case you cannot migrate immediately.
How to use the new versions for testing BEFORE the update
The new versions of the software packages can be selected by using the jun19 tag in modules. For example:
module swap metview/jun19 module load Magics/jun19 module swap eccodes/jun19 module swap python/jun19
How to revert to the old versions in case of problems AFTER the update
Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:
module swap metview/old module load Magics/old module swap eccodes/old module swap python/old
Ecflow change: possible impact after upgrading
This release corrects some faulty triggers expression, that may require validation:
ecflow 4.11.0 or less, these are now errors in 4.12.0 ... family f1 task t1 trigger f1 == aborted task t2 trigger ./f1 == aborted | corrected triggers in ecflow 4.12.0 ... family f1 task t1 trigger ../f1 == aborted task t2 trigger ../f1 == aborted |
---|
If your suite contains wrongly defined triggers like the one on the left, they will never be satisfied and therefore the tasks or families depending on them will not run.
It is highly recommended that whenever you migrate to version 4.12 or beyond from an earlier release, you run the checks below. This will highlight problems early, and save you a lot of debug time.
This can be done before we migrate.
If you use python to generate your suite definitions, then simple make sure you run(defs.check())
module load ecflow/
new
...
# build your python suite definitions
def = ...
print defs.check() #
this
will print any problems with triggers and inlimit
Alternatively you can use:
module swap ecflow/
4.12.
0
ecflow_client --get > tmp.def
ecflow_client --load tmp.def check_only #
this
will print any problems with triggers and inlimit
If you have migrated to the new release, to check all your suites run:
|
You have all the details in the ecflow release 4.12.0.
Change list
ECMWF Software
This table summarises the changes in the version of the different packages:
Package | Default version | Release notes | Download page | |
---|---|---|---|---|
Before | After | |||
CodesUI | 1.2.1 | 1.2.4 | CodesUI - Change History | CodesUI - Releases |
ecCodes | 2.9.2 | 2.12.5 | ecCodes Releases | |
ecFlow | 4.11.1 | 4.12.0 | ecFlow Releases | |
Magics | 3.2.2 | 4.0.3 | Latest News | Magics Releases |
Metview | 5.4.1 | 5.5.4 | Metview Releases Metview Bundle | |
ODB API | 0.18.0 | 0.18.1 | Change history | ODB API Release |
quaver | 1.2.5 | 1.2.6 | Quaver Release Notes | - |
ecmp | 1.0.3 | 1.1.1 | ecmp Release Notes | - |
ecCodes is now the official ECMWF encoder and decoder for GRIB and BUFR formats. It supersedes GRIB-API, for the GRIB format, and EMOSLIB, for the BUFR. If you still use any of those, we encourage you to migrate to ecCodes.
Package | Default version | Release notes | Download page | |
---|---|---|---|---|
Before | After | |||
EMOSLIB | 4.5.8 | 4.6.1 | History of EMOSLIB changes | EMOSLIB Releases |
Please see the Release notes for full details of the changes implemented in each package.
GRIB-API is now out of support and not updated any longer. If you are still using it, we strongly encourage you to migrate to ecCodes.
Third-party software
Some of the versions in this list may already be the default for a specific package on a certain platform, so no changes will be made in those cases. After the update, all platforms will have the same defaults for all these packages.