ECMWF is updating the default versions of its software packages and libraries used across all user platforms at the centre as part of the regular update cycle.

Table of contents

Highlights

Software stack ready for next operational IFS cycle

With the upcoming Implementation of IFS cycle 46r1, this set of defaults is ready to handle the data produced, including all the new parameters introduced in this cycle.

Default GNU compiler updated

The GNU GCC compiler is used by default in our Linux Clusters. After this session, version 7.3 of this popular compiler will be used by default, instead of the old 6.3.

Python 3 Ready

A more complete Python 3 deployment has been made, with all major ECMWF packages supporting it: Magics, Metview, ecFlow and ecCodes (in beta). We encourage all Python users to start migrating their programs to Python 3, as the official support for Python 2 will cease at the end of 2019. Please have a look at the Migration to Python 3 documentation page for more details on how to do this migration.

Metview new features

The new Metview 5.5 features a new set of thermodynamic functions and lots of new functionality for the Geopoints format.  It also uses the new interpolation package MIR by default, replacing the old EMOSLIB interpolation library.

See the Magics and Metview latest news for a more detailed list of changes.

Improved ecFlowUI

This ecFlow release introduces the Timeline View in ecFlowUI. This is available as a tab in the Info Panel and provides a graphical view of node state history taken from the log file, allowing greater overview and analysis of node start and run times. It also supports a Duration view mode, displaying the submission and active times. This mode allows you to filter the nodes of your choice. The Timeline View provides the ideal tool to help you identify and optimise the tasks that are taking too much time.

Update of main third-party software package defaults across all platforms

Once a year we take the opportunity to update not only the software we develop at ECMWF, but also the core 3rd party packages and libraries in our software stack. Amongst the affected packages we can find popular tools and libraries like Python, R,  cdo or netCDF 4. See below for the complete list.

Discontinuation of old PGI compilers

Up to now, we have been providing our software libraries built with a legacy PGI 14 compiler for the FORTRAN parts of the code. This helped with the migration from older platforms to new ones where our default compiler of choice has been the GNU GCC. The new versions becoming default now are no longer available for the PGI environment on those platforms where it is present. If you still use it, please consider moving to the default environment. Note that older versions of our software stack will still be available for a bit longer in case you cannot migrate immediately.

How to use the new versions for testing BEFORE the update

The new versions of the software packages can be selected by using the jun19 tag in modules. For example:

module swap metview/jun19
module load Magics/jun19
module swap eccodes/jun19 
module swap python/jun19

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

module swap metview/old
module load Magics/old
module swap eccodes/old 
module swap python/old

Ecflow change: possible impact after upgrading

This release corrects some faulty triggers expression, that may require validation:

...
family f1
   task t1
       trigger f1 == aborted
   task t2
       trigger ./f1 == aborted

...
family f1
   task t1
        trigger ../f1 == aborted
   task t2
        trigger ../f1 == aborted

If your suite contains wrongly defined triggers like the one on the left, they will never be satisfied and therefore the tasks or families depending on them will not run.

It is highly recommended that whenever you migrate to version 4.12 or beyond from an earlier release, you run the checks below. This will highlight problems early, and save you a lot of debug time.

This can be done before we migrate.

•  If you use python to generate your suite definitions, then simple make sure you run(defs.check())

  module load ecflow/ new ... # build your python suite definitions def = ... print defs.check()  #  this will print any problems with triggers and inlimit

  
  

• Alternatively you can use:

  module swap ecflow/ 4.12.0 ecflow_client --get > tmp.def ecflow_client --load tmp.def check_only #  this will print any problems with triggers and inlimit

  
  

If you have migrated to the new release, to check all your suites run:

You have all the details in the ecflow release 4.12.0.

Change list

ECMWF Software

This table summarises the changes in the version of the different packages:

Please see the Release notes for full details of the changes implemented in each package.

Third-party software