Blog

We have released hotfix versions of several packages, enhancing the October 2020 software release.

ecCodes 2.19.1 includes fixes for the S2S suite and IFS quality checks.

FDB was updated to 5.7.1 with an important fix for backward compatibility with indexes generated from previous versions.

Metkit was updated to 1.6.1 with enhanced MARS language support for all N grids supported by Atlas library.

Magics 4.5.1 includes various small fixes with projection, and handling netCDF and a bug fix in the visualisation of rotated GRIB files.

contour_interpolation_ceiling fixed for rotated grids

NetCDF : small fix for X-Section

Metview 5.10.1 includes various small fixes in plotting, geopoints, startup and conda builds.
An updated Metview Bundle contains these new versions. Versions on conda will be updated in the coming days. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

Note:
These versions will become the default at ECMWF. For more information, please see Change of default versions of ECMWF software packages - November 2020

The October 2020 release of our software provides many new features, improvements and fixes!

ecCodes

ecCodes, the heart of GRIB and BUFR handling, brings performance improvements, new parameters, contributions from users and several smaller features/bug fixes.
Please note that ecCodes' Python 2 interface is no longer being developed. Users are strongly advised to migrate to Python 3.
Also the pip package name has changed to 'eccodes'. Please use this command to install the ecCodes Python bindings:

pip install eccodes

Magics

Magics, ECMWF's plotting library, brings more improvements including :

• a major code clean-up has reduced the number of dependencies. The Boost library and Python are not required to install Magics anymore.
• fix issue with logging that was degrading observation plotting speed when used from Python
• fix issue in python when passing single element array.

Metview

Metview brings several new features and functions, such as:

• a new regridding module that exposes many new interpolation methods for expert users
• user interface improvements such as a new preview panel that allows quick inspection of icons
• the long-awaited ability to filter for parameters in the icon editors makes its way into this release
• a whole raft of new Python/Macro functions for thermodynamic computations and plotting utilities
• more - see the release notes!

Metview's new Regrid module brings new possibilities in gridding and transformation of data

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the Metview Bundle. 

CodesUI

CodesUI, ECMWF's standalone examiner for GRIB and BUFR data, adds:

• syntax highlighting for textual GRIB dumps

MARS

The software stack that serves ECMWF MARS Catalogue has been updated and will be soon rolled out to operations, bringing:

• improved ODB observation handling using the new ODC encoder/decoder software
• improved remote access to FDB servers, including preparation a new FDB service dedicated to research experiments.

FDB

FDB, ECMWF's IFS model I/O library and server application, brings changes to its Remote Protocol that will:

• improve the remote access by the MARS client
• provide better user feedback on errors
• provide support for the migration to Bologna by improved multi-lane access to data.

ecBuild

ecBuild, ECMWF's build system, used by all other software packages, brings major improvements to follow modern CMake best practices. In particular, the minimum CMake version requirement by all packages is now 3.12, for the following reasons:

• better support of find_package, including the use of the <package>_ROOT environment variable
• support for object libraries
• improved dependency tracking in Fortran modules

A list of changes that need to be made to migrate to ecBuild 3.4 is available on GitHub.

Please note that because of the build changes, each package must be built with the latest version of the others.

The software versions of this release are available on all ECMWF computer systems as "new" versions. Versions on conda should be available in the coming days. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

The June 2020 release of our software provides many new features, improvements and fixes!

ecCodes, the heart of GRIB and BUFR handling, brings performance improvements, support for the Mercator projection and several smaller features/bug fixes.
Please note that ecCodes' Python 2 interface is no longer being developed. Users are strongly advised to migrate to Python 3.

Magics, ECMWF's plotting library, and  Metview, ECMWF's interactive and batch software for manipulation and plotting of data, combine with the latest ecCodes to bring plotting support, including interactive Cursor Data for GRIBs encoded with various projections such as Mercator and Lambert variants:

Metview brings some more improvements, such as:

• much improved speed of plotting in-memory arrays of data using the Input Visualiser thanks to allowing vectors (Macro language) or numpy arrays (Python) to be used as data
• 
• the last two modules that were originally implemented in Fortran (Spectra and Potential Temperature) have been converted to C++, meaning they will be available to more users
• fixes for macOS
• new Gallery examples
• more - see the release notes!

CodesUI, ECMWF's standalone examiner for GRIB and BUFR data, adds a fix:

• fixed an issue where the BUFR filter could unexpectedly crash

Please note that when building from source, the minimum CMake version requirement is 3.6.

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the Metview Bundle. The versions are available on all ECMWF computer systems as "new" versions. Versions on conda have also been updated. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

Dear GRIB API users,

The GRIB API software package has long been superseded by ecCodes, as
announced in 2017.

Our next step to fully decommissioning the software package is the removal of
the GRIB API documentation hosted in our Confluence site
(https://confluence.ecmwf.int/display/GRIB), which includes all source
releases of the software.

The removal will happen on the 30th June 2020. Users are invited to update
any links to point to the ecCodes documentation instead. The ecCodes
documentation link is: https://confluence.ecmwf.int/display/ECC

If you have any concerns, please contact Software Support as soon as possible.

Kind regards

ECMWF Software Support
software.support@ecmwf.int

We have released hotfix versions of Magics, Metview and ecFlow.

• ecFlow 5.3.1 includes some fixes to ensure a better experience
• Magics 4.3.1 includes an important fix that prevents a crash in Metview's interactive plot window
• Metview 5.8.2 includes a small fix for the creation of an empty geopoints variable.

Since the crash in Metview's plot window was related to the animation frames panel, we took this opportunity to write some documentation to explain how this works and how you can customise it in order to have the metadata you want to see there.

See Animation frames in the Display Window for more information.

An updated Metview Bundle contains these new versions, and Metview 5.8.2 is available on all ECMWF computer systems as module "new". Versions on conda have also been updated. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

The March 2020 release of our software provides many new features, improvements and fixes!

ecCodes, the heart of GRIB and BUFR handling, brings performance improvements, new parameters and several smaller bug fixes/improvements. Please note that ecCodes' Python 2 interface is no longer being developed. Users are strongly advised to migrate to Python 3.

Metview, ECMWF's interactive and batch software for manipulation and plotting of data, brings a number of new features and improvements, such as:

• easier access to the layer metadata in the interactive plot window
• a new set of pre-defined areas in the Geographical View icon
• improvements to the GRIB and BUFR examiners (and hence also in CodesUI)
• improvements to the existing thermodynamic functions
• plus a new function to compute relative humidity from temperature and dewpoint, a new way to control the font sizes in the user interface
• many more, including some bug fixes
• Metview's Python interface also contains some improvements in ease-of-use and some fixes
  
  • please try the Binder link from github!
  • note that version 1.4.0 of the Python bindings require at least Metview 5.8.1

Metview now offers a more streamlined way to view the layer metadata in a plot, plus a large selection of pre-defined areas

Magics, ECMWF's plotting library, adds some smaller improvements:

• new version of PROJ to handle projections (was using PROJ4): Please report any issues.
• improved plotting of ORCA grids (netCDF)
  
• improved layout of the automatic legend : Adding margin option parameters 
• added format option of the grib_info key dataDate <grib_info key='dataDate' format='%d/%m%Y'/>

CodesUI, the standalone combination of Metview's GRIB and BUFR examiners, provides a highly improved initial BUFR message scan (about 50-100 times faster than before), a filter bar for the GRIB namespace list and extra information about BUFR elements.

Please note that when building from source, the minimum CMake version requirement is 3.6.

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the March 2020 Metview Bundle. The versions are available on all ECMWF computer systems as "new" versions. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

Metview has been available on conda for a while now, but it was a full installation including the graphical user interface. This could cause problems in environments where key libraries for interactive applications are missing.

Now, there is a new conda module, "metview-batch", which will install a batch-only version of Metview. This includes everything from a standard Metview installation except for the user interface. This makes for a smaller and therefore faster installation on conda. It also allows Metview to run in environments that are not intended for user interfaces to run.

Using this new module, a link to Binder has been set up. Now, if you go to Metview's Python interface on github, you'll find a button that will launch a Binder session for you to interactively try out our example Jupyter Notebooks without even installing Metview on your system!

Just click on the "launch binder" button and Binder will spin up a virtual environment with access to our expanding set of notebooks. Click on any .ipynb file to open it - you can edit and run your copy of the code from your web browser! Note that Binder is a free service, and its response may not be immediate.

To install this smaller version of Metview, from a conda environment type:

conda install metview-batch -c conda-forge

As a reminder, the full version of Metview is also available on conda:

conda install metview -c conda-forge

First of all a very Happy New Year to ye all!

The January 2020 release of our software provides many fixes and improvements!

ecCodes, the heart of GRIB and BUFR handling, brings a performance improvement in its Python 3 interface, plus improved support for Microsoft Windows and several smaller improvements. Please note that ecCodes' Python 2 interface is no longer being developed. Users are strongly advised to migrate to Python 3.

Metview, ECMWF's interactive and batch software for manipulation and plotting of data, brings a performance improvement in its Python interface, plus a number of bug fixes

Magics, ECMWF's plotting library, adds some smaller improvements

CodesUI, the standalone combination of Metview's GRIB and BUFR examiners, has a fix for a particular case of BUFR filtering

Please note that when building from source, the minimum CMake version requirement is 3.6.

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the January 2020 Metview Bundle. The versions are available on all ECMWF computer systems as "new" versions. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

The November 2019 release of our software provides many fixes and improvements!

ecCodes, the heart of GRIB and BUFR handling, adds support for new GRIB data e.g. Copernicus regional reanalysis and S2S ocean parameters. It also includes the latest versions of GRIB/BUFR tables/templates released by the WMO in early November.

Metview, ECMWF's interactive and batch software for manipulation and plotting of data, brings some small fixes for improved NetCDF support, better titles on FLEXPART plots and more stable RPM installations.

Magics, ECMWF's plotting library, adds some small fixes including for wind plotting, plus improved plotting of administrative boundaries.

Please note that when building from source, the minimum CMake version requirement is 3.6.

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the November 2019 Metview Bundle. The versions are available on all ECMWF computer systems as "new" versions. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

Version 4.17.0 of ecFlow, ECMWF's workflow management software has been released. It brings a number of improvements in the GUI including improved rendering of date and time attributes that are holding a node, and the ability to kill consumers of a limit directly from the limit's dialogue. See ecFlowUI Release Notes for full details. Among the improvements in the server are an update to the 'repeat date' generated variables, which can now be correctly referenced from trigger expressions for fine-tuned date-based triggering. Another fix in the server allows child tasks that are scheduled to run on a particular day to continue running even if their predecessor ran over the midnight boundary.

The October 2019 release of our software provides many fixes, improvements and new features!

ecCodes, the heart of GRIB and BUFR handling, brings its usual support for new GRIB parameters along with numerous fixes for extra stability. There are performance improvements for decoding/encoding GRIB data in a multi-threaded program as well as new options for the grib_ls and grib_dump tools.

Metview, ECMWF's interactive and batch software for manipulation and plotting of data, brings several new Macro/Python functions, including ml_to_hl(), which converts GRIB data from model levels to height above ground or sea level. Also in this release is an improved inline Help sidebar in the Code Editor.

CodesUI, the standalone combination of Metview's GRIB and BUFR examiners, added new tab called Descriptors to show all the expanded descriptors (metadata only) of the given message.

Magics, ECMWF's plotting library, adds some smaller fixes.

Metview's Code Editor offers a list of available functions and can open a web browser tab to the right place in the documentation for each one

Metview, Magics and ecCodes are all now available on conda through the conda-forge channel. Another way to get the binary installations is via RPM, e.g. through the links here.

Please note that when building from source, the minimum CMake version requirement is 3.6.

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the October 2019 Metview Bundle. The versions are available on all ECMWF computer systems as "new" versions. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

This release brings many exciting changes! Metview brings a number of improvements to its user interface, including improved inline help in the Code Editor and a new way to drop icons. In addition to many improvements to the Macro/Python functions, we also added the much-requested feature to allow the plotting of user-supplied shapefiles as part of the background. If you dig around the documentation, you'll also find some new pages such as the Icon Reference and the Data Types landing pages and 8 new Gallery examples, all designed to make Metview's capabilities more accessible.

ecCodes introduces a new Python 3 interface, available on PyPi (pip3 install eccodes-python). This is the way to install the Python 3 bindings; legacy Python 2 support is still available through the bindings that are built as part of the ecCodes binary package.
Also ecCodes now incorporates the contribution from Météo-France to support limited area model fields for AROME and ALADIN.

Metview can now plot a user-supplied shapefile in a geographic plot

Please note that we have upgraded the minimum CMake version requirement to 3.6.

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the July 2019 Metview Bundle. All these packages use the same build system based on CMake - simplifying and harmonising the installation experience. The versions are available on all ECMWF computer systems as "new" versions. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

With the February 2019 synchronised release, for the first time we make the popular ECMWF software packages ecCodes and Magics available under the Windows operating system. Our partner Daniel Tipping from Old Reliable Tech made the necessary changes in both packages in order to compile under Windows and packaged them with the widely used pip and conda package managers of the Python community. This development allows users to decode and visualise BUFR and GRIB data on Windows using these tools.

Windows is not a platform that is used for technical work at ECMWF. Therefore we will not be able to support this platform as well as we do for others, such as Linux. If you find problems, we invite the Windows user community to use our new set-up on GitHub to fork/contribute and issue pull request changes to ecCodes and Magics. Old Reliable Tech has set up automatic tests and builds for Windows on GitHub, which should ensure continued releases for this new platform.

Please let us have your feedback on this work!

Snapshots showing Magics visualising GRIB fields decoded by ecCodes (right) and Magics showing NetCDF data (left). Both within Jupyter notebooks running under Windows.

This month's software release sees many major changes. This release should bring many improvements for Python users, with core packages, such as ecCodes and Magics now being accessible through pip and conda. This also includes now the Windows operating system. With version 4.0 of Magics the Python interface is now not part of the the core software package anymore. Users should use pip or conda to install the Python extensions.  This has the benefit of allowing support for using multiple Python versions at the same time. Magics also contains improvements for running with Jupyter notebooks. For examples see our Magics Jupyter notebook examples.

This is the first release in which Metview uses the new interpolation package MIR. We recommend users to test this version carefully before introducing this in production use. The previous interpolation through Emoslib is still possible. We welcome any feedback to this change. Metview 5.5 also features a new set of thermodynamic functions and lots of new functionality for the Geopoints format.

New features of Metview: compute and plot a parcel path in a thermodynamic diagram (left) & 
functions based on the horizontal derivatives of regular lat-lon fieldsets have been added (right)

Metview is built at ECMWF with all the versions of our libraries listed in the table below, and all are included in the February 2019 Metview Bundle. All these packages use the same build system based on CMake - simplifying and harmonising the installation experience. The versions are available on all ECMWF computer systems as "new" versions. If you encounter any issues please feel free to send feedback to Software.Support@ecmwf.int.

The newly released ecFlow 4.12.0 introduces a number of new features (ecflow release 4.12.0).

In the future, support for Python 2 will be dropped and only Python 3 will be supported. To aid this effort, ecFlow now supports both Python 2 and 3 modules in the same release. This functionality can be enabled if ecFlow is built using CMake 3.12.0 or above. At ECMWF, ecFlow is now built with support for both Python 2.7 and Python 3.6.

This release also introduces the Timeline View in ecFlowUI. This is available as a tab in the Info Panel and provides a graphical view of node state history taken from the log file, allowing greater overview and analysis of node start and run times. It also supports a Duration view mode, displaying the submission and active times. This mode allows you to filter the nodes of your choice. The Timeline View provides the ideal tool to help you identify and optimise the tasks that are taking to much time.

This release  allows us to list, the set of nodes that depend on a Event.(i.e. trigger references to an event).   First select the event in ecflowUI, then with RMB select  'Edit ...'

The script location algorithm (i.e. how the server finds your .ecf/.sms files) is now more flexible. Locating ‘.ecf’ file

This release corrects some faulty triggers expression, that may require validation.

It is highly recommended that whenever you migrate from one release to another, you run the checks(see below) on the new release.

This will highlight problems early, and save you a lot of debug time.

This can be done before we migrate.

•  If you use python to generate your suite definitions, then simple make sure you run(defs.check())

  module load ecflow/new ... # build your python suite definitions def = ... print defs.check()  # this will print any problems with triggers and inlimit

• Alternatively you can use:

  module swap ecflow/4.12.0 ecflow_client --get > tmp.def ecflow_client --load tmp.def check_only # this will print any problems with triggers and inlimit

• If you have migrated to the new release, to check all your suites run:

  module load ecflow/4.12.0 ecflow_client --check=_all_   # this will print any problems with triggers and inlimits