Versions Compared

Old Version 8

changes.mady.by.user Daniel Varela Santoalla

Saved on

compared with

New Version 9

changes.mady.by.user Axel Bonet

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

The ecFlow user interface (ecFlowview) uses the same technology and user paradigm as XCDP. The graphical user interface (GUI) is X-Windows based. This places limits on the machines where user interface can be run, since the X windows libraries will require a separate installation. This will be replaced in the near future, with a more intuitive and up to date GUI.
When ecFlow is released the online documentation and tutorial can be accessed from: httphttps://wwwsoftware.ecmwf.int/publicationswiki/manualsdisplay/ecflow/ECFLOW
We welcome feedback on any aspect of ecFlow. Suggestions for further developments will be assessed and, where appropriate, they will be prioritised and fed into future versions.

Box A. Correct by construction

The following examples show usage of the new Python API. The API supports a correct by construction approach. For example, adding tasks of the same name at the same level will throw a RuntimeError exception:

Example 1

Code Block
languagepython
import ecflow
defs = ecflow.Defs()
suite = defs.add_suite("s1")
suite.add_task("t1")
suite.add_task("t1") # RuntimeError exception thrown
>> RuntimeError: Add Task failed: A task of name 't1' already exist on node SUITE:/s1


Example 2
Adding dependencies, such as dates, are also checked:

Code Block
languagepython
import ecflow
defs = ecflow.Defs()
suite = defs.add_suite("s1")
task t1 = suite.add_task("t1")
t1.add_date(1,14,2007) # day,month,year, month is not valid
>> IndexError: Invalid Date(day,month,year): the month >=0 and month <= 12, where 0 means wild card

Box B. Checking

Expression checking

Some checking has to be deferred until the definition is fully defined. Here is a simple example showing the checking of trigger expressions:

Code Block
languagepython
import ecflow
defs = ecflow.Defs() 
suite = defs.add_suite("s1");
suite.add_task("t2") 
suite.add_task("t1").add_trigger("t2 == active)") 
assert len(defs.check()) != 0, "Expected Error: miss-matched brackets in expression."

 

Job checking

Job creation is the process of locating an '.ecf' script corresponding to a task and then generating a job file. This can be checked before a definition is loaded into the server using the Python API.

Code Block
languagepython
# Generate jobs for the ALL tasks in the definition given by variable 'defs' 
# and print errors to standard out. 
import ecflow 
defs = ecflow.Defs() 
suite = defs.add_suite("s1")
suite.add_task("t1") 
print defs.check_job_creation()



Job control provides additional functionality to control which nodes are generated and control over the directory used for job generation.

Dead lock checking

Simulation allows a suite definition to be checked without the need for scripts or a server. By default the simulation will run for a year before quitting. This can take a couple a seconds to a few minutes depending on the complexity of the suite definition. However, it is most useful where we have a definition which is known to complete. Here is an example which will cause a deadlock that is detectable by the simulator.

Code Block
languagepython
import os from ecflow import *
defs = Defs() 
suite = defs.add_suite("dead_lock") 
fam = suite.add_family("family") 
fam.add_task("t1").add_trigger("t2 == complete") 
fam.add_task("t2").add_trigger("t1 == complete")
theResult = defs.simulate()
assert len(theResult) != 0, "Expected simulation to return errors, but found none" print theResult
os.remove("defs.depth") # provides reason why simulation could not complete 
os.remove("defs.flat")  # provides reason why simulation could not complete



Early checking of the suite definition will help to speed up the development of suites.

Box C. Boost C++ libraries used in ecFlow

  • Boost-ASIO library is used to provide the core of the client-server implementation. It also provides a deadline timer for polling and support for time outs.
  • Boost-Python library enables seamless interoperability between C++ and the Python programming language.
  • Boost-Program options are used to parse the client and server program options, and they provide the corresponding help strings.
  • Boost-Spirit provides the parsing for expressions. The abstract syntax tree is then created from the spirit nodes.
  • Boost-Test library is used for unit and regression tests of the C++ and Python code.
  • Boost-Date-Time library is used in the suite calendar, time based attributes and polling.
  • Boost-File System library is used for file queries in a platform independent manner.