Page tree
Skip to end of metadata
Go to start of metadata
What is ecFlow



ecFlow is a client/server workflow package that enables users to run a large number of programs (with dependencies on each other and on time) in a controlled environment. It provides reasonable tolerance for hardware and software failures, combined with restart capabilities. It is used at ECMWF to run all our operational suites across a range of platforms.

ecFlow submits tasks (jobs) and receives acknowledgments from the tasks when they change status and when they send events, using child commands embedded in your scripts. ecFlow stores the relationships between tasks, and is able to submit tasks dependant on triggers, such as when a given task changes its status, for example when it finishes.  

ecFlow functionality is provided by the following executables and shared libraries

  • ecflow_client: This executable is a command-line program; it is used for all communication with the server. This executable implements the Command Level Interface (CLI). The bulk of this functionality is also provided by the python API
  • ecflow_ui: This is a specialized GUI client that monitors and visualizes the node tree hierarchy. Based on QT.
  • ecflow_server: This executable is the server. It is responsible for scheduling the jobs and responding to ecflow_client requests.
  • ecflow.so:    Python interface These shared libraries provide the python API for creating the suite definition and communication with the server.

ecFlow runs as a server receiving requests from clients. CLI, GUI, and suite jobs are clients. Communication is based on TCP/IP. Note that ecFlow is a scheduler and is not a queuing system such as NQS, SGE, Load leveler, or PBS. However, it can submit to queuing systems.

History

For almost three decades ECMWF used SMS (Supervisor Monitoring Scheduler) package to control the workflow for the Centre's operational models and systems. Written at ECMWF, it allows the design, submission, and monitoring of jobs both in the Research and Operations Departments, and provides common tools for scientists, analysts, and operators to cooperate. A large number of organizations also use SMS (from both Member and non-Member States). Development and support of SMS has now stopped. 

The replacement, ecFlow 4, has superseded SMS. It has been used for over a decade. It is a complete rewrite using an object-oriented methodology and modern standardized components; it acts as a comprehensive replacement for SMS. The rewrite helps improve maintainability, allows easier modification, and introduces object orientated features. The proprietary script language used by SMS, CDP, has been replaced by Python.

The development of ecflow 4 has now stopped. But support is still provided. It has been replaced by ecflow 5

You can find an introduction to ecFlow in the ECMWF Newsletter article  (Autumn 2011), starting on page 30.







Latest News

Go to Latest news to access older news items

 

Report a bug or issue

Please send an email to software.support@ecmwf.int or go to Issues if you have any suggestions for improvements or have discovered a bug with this software package.

Blog Posts

Releases

  File Modified
File ecFlow-5.5.3-Source.tar.gz Sep 24, 2020 by Application deploy
File ecFlow-5.5.2-Source.tar.gz Sep 10, 2020 by Application deploy
File ecFlow-5.5.1-Source.tar.gz Jul 09, 2020 by Application deploy
File ecFlow-5.5.0-Source.tar.gz Jul 03, 2020 by Application deploy
File ecFlow-5.4.0-Source.tar.gz May 06, 2020 by Application deploy
ZIP Archive ecFlow-5.3.1-Source.tar.gz production release Apr 02, 2020 by Avi Bahra
File ecFlow-4.17.2-Source.tar.gz production release Mar 03, 2020 by Application deploy
File ecFlow-5.3.0-Source.tar.gz production release Mar 03, 2020 by Application deploy
File ecFlow-5.2.3-Source.tar.gz production release Feb 18, 2020 by Application deploy
File ecFlow-5.2.2-Source.tar.gz production release Feb 14, 2020 by Application deploy
File ecFlow-5.2.1-Source.tar.gz production release Jan 29, 2020 by Application deploy
File ecFlow-5.2.0-Source.tar.gz production release Jan 20, 2020 by Application deploy
File ecFlow-4.17.1-Source.tar.gz production release Jan 08, 2020 by Application deploy
File ecFlow-4.17.0-Source.tar.gz production release Oct 10, 2019 by Application deploy
File ecFlow-5.1.0-Source.tar.gz production release Oct 02, 2019 by Application deploy
File ecFlow-4.16.0-Source.tar.gz production release Aug 15, 2019 by Application deploy
File ecFlow-4.15.0-Source.tar.gz production release Jun 24, 2019 by Application deploy
File ecFlow-4.14.0-Source.tar.gz production release Mar 27, 2019 by Application deploy
File ecFlow-4.13.0-Source.tar.gz production release Feb 21, 2019 by Application deploy
File ecFlow-4.12.0-Source.tar.gz production release Jan 17, 2019 by Application deploy
File ecFlow-4.11.1-Source.tar.gz production release Oct 23, 2018 by Application deploy
File ecFlow-4.11.0-Source.tar.gz production release Oct 16, 2018 by Application deploy
ZIP Archive ecFlow-4.10.0-Source.tar.gz production release Jul 23, 2018 by Avi Bahra
File ecFlow-4.9.0-Source.tar.gz production release Apr 05, 2018 by Application deploy
ZIP Archive ecFlow-4.8.0-Source.tar.gz production release Jan 16, 2018 by Avi Bahra
ZIP Archive ecFlow-4.7.1-Source.tar.gz production release(hotfix) Nov 07, 2017 by Avi Bahra
ZIP Archive ecFlow-4.7.0-Source.tar.gz production release Oct 11, 2017 by Avi Bahra
ZIP Archive boost_1_53_0.tar.gz Boost libs for ecflow > 4.0.0 Nov 14, 2013 by Avi Bahra

Binary releases

Binary package notes

  • At the moment these packages are still experimental. Please report any issue with them to Software.Support@ecmwf.int or via JIRA to http://software.ecmwf.int/issues/browse/SUP .
  • Some components of ecflow are not available in some platforms. A minimum of python 2.5 is required to compile the python bindings and since some distributions ship with an older version we have chosen not to build the python bindings for those. In addition, ecflowview is not available for all platforms.
  • We have a limited availability of build platforms for ecflow. If you want to have a binary for any platform not listed here you can tell us by sending an email to Software.Support@ecmwf.int or reporting it via JIRA in http://software.ecmwf.int/issues/browse/SUP. We can not cover all possible platforms but we are still interested on hearing from you
  • Packages from version 4.0.0-1 onwards have divided the client and server parts of ecflow into two different packages: ecflow for the client side and ecflow-server for the server side. Please keep in mind that for a fully functional server you need both packages and that client and server versions must be matched.

Current and legacy list of packages

 

 

 

 

 

Recently Updated