Page History

Horizontal Navigation Bar

Button Group

Button Hyperlink

title	Previous
type	standard
url	https://

software

confluence.ecmwf.int/

wiki/

display/ECFLOW/

Labels

Autoarchive+and+Autorestore

Button Hyperlink

title	Up
type	standard
url	https://

software

confluence.ecmwf.int/

wiki/

display/ECFLOW/Advanced+Topics

Button Hyperlink

title	Next
type	standard
url	https://

software

confluence.ecmwf.int

/wiki

/display/ECFLOW/

LimitsIt is sometimes useful to repeat the same task or family several times,

looping on a specific value. You can do that by defining a repeat attribute.

You can iterate over sequences of:

strings
integers
dates

Alias

Sometimes tasks don't run as expected, and we want to get a notification when this is the case.

For this, we use the late attribute.

A node can only have one late attribute. The late attribute only applies to a task. You can define it on a Suite/Family in which case it will be inherited. Any late defined lower down the hierarchy will override the aspect(submitted, active, complete) defined higher up.

-s submitted: The time node can stay submitted (format [+]hh:mm). submitted is always relative, so + is simply ignored, if present. If the node stays submitted longer than the time specified, the late flag is set
-a Active: The time of day the node must have become active (format hh:mm). If the node is still queued or submitted, the late flag is set
-c Complete: The time node must become complete (format {+}hh:mm). If relative, time is taken from the time the node became active, otherwise the node must be complete by the time given.

Code Block

title	Late example

task t1 
   late -s +00:15 -a 20:00 -c +02:00

This is interpreted as, the node can stay submitted for a maximum of 15 minutes, and it must become active by 20:00 and the runtime must not exceed 2 hours.

For the purposes of this tutorial, we will add a late attribute for the runtime only

A sequence of integers or dates is created by specifying the

first and last element, with an optional increment (the default is one).

An ecFlow variable, whose name corresponds to the name of the repeat,

will be generated. This can be used in scripts or trigger expressions

.

Ecf Script

We will add a new task /test/f4/f5f6/t1.

Create new ecf script file $HOME/course/test/f4f6/f5/t1.ecf to use these variables for which we want to be late.

Code Block

language	bash
title	$HOME/course/test/f6/t1.ecf

%include <head.h>
ecflow_client --label=info "My name is %NAME%" "My value is %VALUE%" "My date is %DATE%"echo "I will now sleep for %SLEEP% seconds"
sleep %SLEEP%
%include <tail.h>

Text

Let us modify the suite definition file again

Code Block

# Definition of the suite test.
suite test
 edit ECF_INCLUDE "$HOME/course"    # replace $HOME with the path to your home directory
 edit ECF_HOME    "$HOME/course"

 family f4f6
     edit edit SLEEP 2120
     repeat stringtask NAMEt1
 a b c d e f
     late family f5
         repeat integer VALUE 1 10
         task t1
             repeat date DATE 19991230 20000105-c +00:01 # set late flag if task take longer than a minute
 endfamily
endsuite

Python

Code Block

language	py
title	$HOME/course/test.py

import os
from ecflow import Defs,Suite,Family,Task,Edit,Trigger,Complete,Event,Meter,Time,Day,Date,Label, \
             label info ""
     endfamily
 endfamily
endsuite

Python

#!/usr/bin/env python2.7 import os import ecflow

RepeatString,RepeatInteger,RepeatDate,InLimit,Limit,Late
        
def create_family_

f4

f6():

f4

return

= ecflow.

Family("

f4

f6"

)

f4.add_variable("SLEEP",

2)

f4.add_repeat(

ecflow.RepeatString("NAME",

["a",

"b",

"c",

"d", "e", "f" ] ) )

 Edit(SLEEP=120),

f5

=

f4.add_family("f5")

f5.add_repeat( ecflow.RepeatInteger("VALUE

Task("t1",

1,

10)

)

t1

=

f5.add_task("t1")

t1.add_repeat(

ecflow.RepeatDate("DATE",

20101230, 20110105) ) t1.add_label("info", "") return f4 print

 Late(complete='+00:01'))) # set late flag if task t1 takes longer than a minute

print("Creating suite definition")

defs = ecflow.Defs() suite = defs.add_suite("test") suite.add_variable("ECF_INCLUDE",


home = os.path.join(os.getenv("HOME"),

"course")

) suite.add_variable("ECF_HOME", os.path.join(os.getenv("HOME"), "course")) suite.add_family(


defs = Defs( 
        Suite("test",
            Edit(ECF_INCLUDE=home,ECF_HOME=home),
            create_family_

f4

f6()

))
print(defs)

defs



print

("Checking job creation: .ecf -> .job0")


print

(defs.check_job_creation())

print("Checking trigger expressions")
assert len(defs.check()) == 0,defs.check() 

print("Saving definition to file 'test.def'")
defs.save_as_defs("test.def")

What to do

Type in the changes
Replace the suite definition
How many times will /test/f4/f5/t1 run?
In ecflow_ui , try to modify the values of a repeat

Run the suite, you should see the task late flag set in ecflow_ui
When the job completes, if you re-queue family node f6 or task t1, it will clear the late flag. The late flag can also be cleared manually, select task t1, then with Right Mouse Button, → Special → Clear late flag

Button Group

Button Hyperlink

title	Previous
type	standard
url	https://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/LabelsAutoarchive+and+Autorestore

Button Hyperlink

title	Up
type	standard
url	https://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/Advanced+Topics

Button Hyperlink

title	Next
type	standard
url	https://softwareconfluence.ecmwf.int/wiki/display/ECFLOW/LimitsAlias

Space shortcuts

Page tree

Versions Compared

Old Version 2

New Version Current

Key

Ecf Script

Text

Python

Python

What to do