Skip to end of metadata
Go to start of metadata

ECMWF is updating the default versions of its software packages and libraries used across all user platforms at the centre as part of the regular update cycle.

When?

The changes will take place on  5 June 2019 10:00 - 10:30 UTC

What platforms are affected?

All user platforms at the centre will see their default additional software stack updated:

  • All clusters in our High Performance Computing Facility, mainly CCA and CCB
  • ECGATE
  • Linux Clusters
  • Linux Desktops

Do I need to do anything?

We strongly recommend you test that your programs, scripts and suites work with the new versions of any software or library affected before the change happens.

In particular, if using ecFlow, please make sure you check the triggers in your suites are well defined as described below.

Table of contents

Highlights

Software stack ready for next operational IFS cycle

With the upcoming Implementation of IFS cycle 46R1, this set of defaults is ready to handle the data produced, including all the new parameters introduced in this cycle.

Default GNU compiler updated

The GNU GCC compiler is used by default in our Linux Clusters. After this session, version 7.3 of this popular compiler will be used by default, instead of the old 6.3.

Python 3 Ready

A more complete Python 3 deployment has been made, with all major ECMWF packages supporting it: Magics, Metview, ecFlow and ecCodes (in beta). We encourage all Python users to start migrating their programs to Python 3, as the official support for Python 2 will cease at the end of 2019. Please have a look at the Migration to Python 3 documentation page for more details on how to do this migration.

Metview new features

The new Metview 5.5 features a new set of thermodynamic functions and lots of new functionality for the Geopoints format.  It also uses the new interpolation package MIR by default, replacing the old EMOSLIB interpolation library.

See the Magics and Metview latest news for a more detailed list of changes.

Improved ecFlowUI

This ecFlow release introduces the Timeline View in ecFlowUI. This is available as a tab in the Info Panel and provides a graphical view of node state history taken from the log file, allowing greater overview and analysis of node start and run times. It also supports a Duration view mode, displaying the submission and active times. This mode allows you to filter the nodes of your choice. The Timeline View provides the ideal tool to help you identify and optimise the tasks that are taking too much time.

Update of main third-party software package defaults across all platforms

Once a year we take the opportunity to update not only the software we develop at ECMWF, but also the core 3rd party packages and libraries in our software stack. Amongst the affected packages we can find popular tools and libraries like Python, R,  cdo or netCDF 4. See below for the complete list.

Discontinuation of old PGI compilers

Up to now, we have been providing our software libraries built with a legacy PGI 14 compiler for the FORTRAN parts of the code. This helped with the migration from older platforms to new ones where our default compiler of choice has been the GNU GCC. The new versions becoming default now are no longer available for the PGI environment on those platforms where it is present. If you still use it, please consider moving to the default environment. Note that older versions of our software stack will still be available for a bit longer in case you cannot migrate immediately.

How to use the new versions for testing BEFORE the update

Users are strongly encouraged to test their software applications and data processing chain with the new versions of the various software packages before the date of the change to the default versions.

The new versions of the software packages can be selected by using the jun19 tag in modules. For example:

module swap metview/jun19
module load Magics/jun19
module swap eccodes/jun19 
module swap python/jun19

How to revert to the old versions in case of problems AFTER the update

Users that experience problems with the new default software packages after the update will be able to revert to the old default versions. These can be selected with the commands:

module swap metview/old
module load Magics/old
module swap eccodes/old 
module swap python/old

Please let ECMWF know if you need to revert to the old version of any of the packages after the update of the defaults has taken place. 

Ecflow change: possible impact after upgrading

This release corrects some faulty triggers expression, that may require validation:

ecflow 4.11.0 or less, these are now errors in 4.12.0
...
family f1
   task t1
       trigger f1 == aborted
   task t2
       trigger ./f1 == aborted
corrected triggers in ecflow 4.12.0
...
family f1
   task t1
        trigger ../f1 == aborted
   task t2
        trigger ../f1 == aborted

If your suite contains wrongly defined triggers like the one on the left, they will never be satisfied and therefore the tasks or families depending on them will not run.

It is highly recommended that whenever you migrate to version 4.12 or beyond from an earlier release, you run the checks below. This will highlight problems early, and save you a lot of debug time.

This can be done before we migrate.

  •  If you use python to generate your suite definitions, then simple make sure you run(defs.check())

    module load ecflow/ new
    ...
    # build your python suite definitions
    def = ...
    print defs.check()  #  this will print any problems with triggers and inlimit

  • Alternatively you can use:

    module swap ecflow/ 4.12.0
    ecflow_client --get > tmp.def
    ecflow_client --load tmp.def check_only #  this will print any problems with triggers and inlimit

If you have migrated to the new release, to check all your suites run:

module load ecflow/ 4.12.0
ecflow_client --check=_all_   #  this will print any problems with triggers and inlimits

You have all the details in the ecflow release 4.12.0.

Change list

ECMWF Software

This table summarises the changes in the version of the different packages:

ecCodes is now the official ECMWF encoder and decoder for GRIB and BUFR formats. It supersedes GRIB-API, for the GRIB format, and EMOSLIB, for the BUFR. If you still use any of those, we encourage you to migrate to ecCodes.

PackageDefault versionRelease notesDownload page
BeforeAfter
EMOSLIB4.5.84.6.1History of EMOSLIB changesEMOSLIB Releases

Please see the Release notes for full details of the changes implemented in each package.

GRIB-API is now out of support and not updated any longer. If you are still using it, we strongly encourage you to migrate to ecCodes.


Third-party software

Some of the versions in this list may already be the default for a specific package on a certain platform, so no changes will be made in those cases. After the update, all platforms will have the same defaults for all these packages.


  • No labels