ecFlow Python Api
Api
The ecflow module provides the python bindings/api for creating definition structure and communicating with the server.
class ecflow.Alias
Bases:
ecflow.SubmittableA Aliases is create by the GUI or via edit_script command
Aliases provide a mechanism to edit/test task scripts without effecting the suite The Aliases parent is always a Task.Multiple Alias can be added
class ecflow.AttrType
Bases:
Boost.Python.enumSortable attribute type, currently [event | meter | label | limit | variable ]
event = ecflow.AttrType.event
label = ecflow.AttrType.label
limit = ecflow.AttrType.limit
meter = ecflow.AttrType.meter
names = {'variable': ecflow.AttrType.variable, 'limit': ecflow.AttrType.limit, 'event': ecflow.AttrType.event, 'meter': ecflow.AttrType.meter, 'label': ecflow.AttrType.label}
values = {1: ecflow.AttrType.event, 2: ecflow.AttrType.meter, 3: ecflow.AttrType.label, 4: ecflow.AttrType.limit, 5: ecflow.AttrType.variable}
variable = ecflow.AttrType.variable
class ecflow.Autocancel
Bases:
Boost.Python.instanceProvides a way to automatically delete/remove a node which has completed
See autocancel
Constructor:
Autocancel(TimeSlot,relative) TimeSlot single: A time bool relative: Relative to completion. False means delete the node at the real time specified. Autocancel(hour,minute,relative) int hour: hour in 24 hrs int minute: minute <= 59 bool relative: Relative to completion. False means delete the node at the real time specified. Autocancel(days) int days: Delete the node "days" after completion
Usage:
attr = Autocancel( 1,30, true ) # delete node 1 hour and 30 minutes after completion attr = Autocancel( TimeSlot(0,10), true ) # delete node 10 minutes after completion attr = Autocancel( TimeSlot(10,10), false ) # delete node at 10:10 after completion attr = Autocancel( 3 ) # delete node 3 days after completion t1 = Task("t1", Autocancel(2,0,true)) # delete task 2 hours after completiondays((Autocancel)arg1) → bool :
Returns a boolean true if time was specified in days
relative((Autocancel)arg1) → bool :
Returns a boolean where true means the time is relative
time((Autocancel)arg1) → TimeSlot :
returns cancel time as a TimeSlot
class ecflow.CheckPt
Bases:
Boost.Python.enumCheckPt is enum that is used to control check pointing in the ecflow_server
- NEVER : Switches of check pointing
- ON_TIME: check point file is saved periodically, specified by checkPtInterval. This is the default.
- ALWAYS : check point file is saved after any state change, not recommended for large definitions
- UNDEFINED : None of the the above, used to provide default argument
ALWAYS = ecflow.CheckPt.ALWAYS
NEVER = ecflow.CheckPt.NEVER
ON_TIME = ecflow.CheckPt.ON_TIME
UNDEFINED = ecflow.CheckPt.UNDEFINED
names = {'ALWAYS': ecflow.CheckPt.ALWAYS, 'NEVER': ecflow.CheckPt.NEVER, 'UNDEFINED': ecflow.CheckPt.UNDEFINED, 'ON_TIME': ecflow.CheckPt.ON_TIME}
values = {0: ecflow.CheckPt.NEVER, 1: ecflow.CheckPt.ON_TIME, 2: ecflow.CheckPt.ALWAYS, 3: ecflow.CheckPt.UNDEFINED}
class ecflow.ChildCmdType
Bases:
Boost.Python.enumChildCmdType represents the different child command s. This type is used as a parameter to the class
ecflow.ZombieAttrChild commands are called within a job file:
ChildCmdType::init corresponds to : ecflow_client --init=<process_id> ChildCmdType::event corresponds to : ecflow_client --event=<event_name | number> ChildCmdType::meter corresponds to : ecflow_client --meter=<meter_name>, <meter_value> ChildCmdType::label corresponds to : ecflow_client --label=<label_name>. <label_value> ChildCmdType::wait corresponds to : ecflow_client --wait=<expression> ChildCmdType::abort corresponds to : ecflow_client --abort=<reason> ChildCmdType::complete corresponds to : ecflow_client --complete
abort = ecflow.ChildCmdType.abort
complete = ecflow.ChildCmdType.complete
event = ecflow.ChildCmdType.event
init = ecflow.ChildCmdType.init
label = ecflow.ChildCmdType.label
meter = ecflow.ChildCmdType.meter
names = {'complete': ecflow.ChildCmdType.complete, 'meter': ecflow.ChildCmdType.meter, 'label': ecflow.ChildCmdType.label, 'init': ecflow.ChildCmdType.init, 'abort': ecflow.ChildCmdType.abort, 'event': ecflow.ChildCmdType.event, 'wait': ecflow.ChildCmdType.wait}
values = {0: ecflow.ChildCmdType.init, 1: ecflow.ChildCmdType.event, 2: ecflow.ChildCmdType.meter, 3: ecflow.ChildCmdType.label, 4: ecflow.ChildCmdType.wait, 5: ecflow.ChildCmdType.abort, 6: ecflow.ChildCmdType.complete}
wait = ecflow.ChildCmdType.wait
class ecflow.Client
Bases:
Boost.Python.instanceClass client provides an interface to communicate with the ecflow_server.:
Client( string host, # The server name. Can not be empty. string port # The port on the server, must be unique to the server ) Client( string host, # The server name. Can not be empty. int port # The port on the server, must be unique to the server ) Client( string host_port, # Expect"s <host>:<port> || <host>@<port> )
The client reads in the following environment variables. For child commands,(i.e. these are commands called in the .ecf/jobs files), these variables are used. For the python interface these environment variable are not really applicable but documented for completeness:
- ECF_NAME <string> : Full path name to the task
- ECF_PASS <string> : The jobs password, allocated by server, then used by server to authenticate client request
- ECF_TRYNO <int> : The number of times the job has run. Used in file name generation. Set to 1 by begin() and re-queue commands.
- ECF_TIMEOUT <int> : Max time in seconds for client to deliver message to main server
- ECF_HOSTFILE <string> : File that lists alternate hosts to try, if connection to main host fails
- ECF_DENIED <any> : Provides a way for child to exit with an error, if server denies connection. Avoids 24hr wait. Note: when you have hundreds of tasks, using this approach requires a lot of manual intervention to determine job status
- NO_ECF <any> : If set exit’s immediately with success. Used to test jobs without communicating with server
The following environment variables are used by the python interface and child commands
- ECF_HOST <string> : The host name of the main server. defaults to ‘localhost’
- ECF_PORT <int> : The TCP/IP port to call on the server. Must be unique to a server
The ECF_HOST and ECF_PORT can be overridden by using the Constructor or set_host_port() member function. For optimal usage it is best to reuse the same Client rather than recreating for each client server interaction By default the Client interface will throw exceptions for error’s.
Usage:
try: ci = Client("localhost:3150") # for errors will throw RuntimeError ci.terminate_server() except RuntimeError, e: print(str(e))alter((Client)arg1, (list)paths, (str)alter_type, (str)attribute_type[, (str)name=''[, (str)value='']]) → None :
- Alter command is used to change the attributes of a node
void alter( (list | string ) paths(s) : A single or list of paths. Path name to the node whose attributes are to be changed string alter_type : This must be one of [ "add" | "change" | "delete" | "set_flag" | "clear_flag" ] string attr_type : This varies according to the "alter_type". valid strings are: add : [ variable,time,today,date,day,label,zombie,late] delete : [ variable,time,today,date,day,label,cron,event,meter,trigger,complete,repeat,limit,inlimit,limit_path,zombie,late] change : [ variable,clock-type,clock-gain,event,meter,label,trigger,complete,repeat,limit-max,limit-value,late] set_flag and clear_flag: [ force_aborted | user_edit | task_aborted | edit_failed | ecfcmd_failed | no_script | killed | migrated | late | message | complete | queue_limit | task_waiting | locked | zombie ] string name : used to locate the attribute, when multiple attributes of the same type, optional for some.i.e.. when changing, attributes like variable,meter,event,label,limits string value : Only used when "changing" a attribute. provides a new value )Exceptions can be raised because:
- absolute_node_path does not exist.
- parsing fails
The following describes the parameters in more detail:
add variable variable_name variable_value add time format # when format is +hh:mm | hh:mm | hh:mm(start) hh:mm(finish) hh:mm(increment) add today format # when format is +hh:mm | hh:mm | hh:mm(start) hh:mm(finish) hh:mm(increment) add date format # when format dd.mm.yyyy, can use "*" to indicate any day,month, or year add day format # when format is one of [ sunday,monday,tuesday,wednesday,friday,saturday ] add zombie format # when format is one of <zombie-type>:<child>:<server-action>|<client-action>:<zombie-lifetime> # <zombie-type> := [ user | ecf | path ] # <child> := [ init, event, meter, label, wait, abort, complete ] # <server-action> := [ adopt | delete ] # <client-action> := [ fob | fail | block(default) ] # <zombie-lifetime>:= lifetime of zombie in the server # example # add zombie :label:fob:0 # fob all child label request, & remove zombie as soon as possible delete variable name # if name is empty will delete -all- variables on the node delete time name # To delete a specific time, enter the time in same format as show above, # or as specified in the defs file # an empty name will delete all time attributes on the node delete today name # To delete a specific today attribute, enter in same format as show above, # or as specified in the defs file. # an empty name will delete all today attributes on the node delete date name # To delete a specific date attribute, enter in same format as show above, # or as specified in the defs file # an empty name will delete all date attributes on the node delete day name # To delete a specific day attribute, enter in same format as show above, # or as specified in the defs file # an empty name will delete all day attributes on the node delete cron name # To delete a specific cron attribute, enter in same as specified in the defs file # an empty name will delete all cron attributes on the node delete event name # To delete a specific event, enter name or number # an empty name will delete all events on the node delete meter name # To delete a specific meter , enter the meter name # an empty name will delete all meter on the node delete label name # To delete a specific label , enter the label name # an empty name will delete all labels on the node delete limit name # To delete a specific limit , enter the limit name # an empty name will delete all limits on the node delete inlimit name # To delete a specific inlimit , enter the inlimit name # an empty name will delete all inlimits on the node delete limit_path limit_name limit_path # To delete a specific limit path delete trigger # A node can only have one trigger expression, hence the name is not required delete complete # A node can only have one complete expression, hence the name is not required delete repeat # A node can only have one repeat, hence the name is not required change variable name value # Find the specified variable, and set the new value. change clock_type name # The name must be one of "hybrid" or "real". change clock_gain name # The gain must be convertible to an integer. change clock_sync name # Sync suite calendar with the computer. change event name(optional ) # if no name specified the event is set, otherwise name must be "set" or "clear" change meter name value # The meter value must be convertible to an integer, and between meter min-max range. change label name value # sets the label change trigger name # The name must be expression. returns an error if the expression does not parse change complete name # The name must be expression. returns an error if the expression does not parse change limit_max name value # Sets the max value of the limit. The value must be convertible to an integer change limit_value name value # Sets the consumed tokens to value. The value must be convertible to an integer change repeat value # If the repeat is a date, then the value must be a valid YMD ( ie. yyyymmdd) # and be convertible to an integer, additionally the value must be in range # of the repeat start and end dates. Like wise for repeat integer. For repeat # string and enum, the name must either be an integer, that is a valid index or # if it is a string, it must correspond to one of enum"s or strings listUsage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.alter("/suite/task","change","trigger","b2 == complete") except RuntimeError, e: print(str(e))
alter( (Client)arg1, (str)abs_node_path, (str)alter_type, (str)attribute_type [, (str)name=’’ [, (str)value=’‘]]) -> None
begin_all_suites((Client)arg1[, (bool)force=False]) → int :
Begin playing all the suite s in the ecflow_server
Note
using the force option may cause zombie s if suite has running jobs
void begin_all_suites( [(bool)force=False] : bypass the checks for submitted and active jobs )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.begin_all_suites() # begin playing all the suites ci.begin_all_suites(True) # begin playing all the suites, by passing checks except RuntimeError, e: print(str(e))
begin_suite((Client)arg1, (str)suite_name[, (bool)force=False]) → int :
Begin playing the chosen suite s in the ecflow_server
Note
using the force option may cause zombie s if suite has running jobs
void begin_suite string suite_name : begin playing the given suite [(bool)force=False] : bypass the checks for submitted and active jobs )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.begin_suite("/suite1") # begin playing suite "/suite1" ci.begin_suite("/suite1",True) # begin playing suite "/suite1" bypass any checks except RuntimeError, e: print(str(e))
ch_add((Client)arg1, (int)arg2, (list)arg3) → None :
- Add a set of suites, to an existing registered handle
When dealing with large definitions, where a user is only interested in a small subset of suites, registering them, improves download performance from the server. Registered suites have an associated handle.
integer ch_add( integer handle : the handle obtained after ch_register list suite_names : list of strings representing suite names ) integer ch_add( list suite_names : list of strings representing suite names )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.ch_register(True,[]) # register interest in any new suites ci.ch_add(["s1","s2"]) # add suites s1,s2 to the last added handle except RuntimeError, e: print(str(e))
ch_add( (Client)arg1, (list)arg2) -> None
ch_auto_add((Client)arg1, (int)arg2, (bool)arg3) → int :
- Change an existing handle so that new suites can be added automatically
When dealing with large definitions, where a user is only interested in a small subset of suites, registering them, improves download performance from the server. Registered suites have an associated handle.
void ch_auto_add( integer handle, : the handle obtained after ch_register bool auto_add_new_suite : automatically add new suites, this handle when they are created ) void ch_auto_add( bool auto_add_new_suite : automatically add new suites using handle on the client )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.ch_register(True,["s1","s2","s3"]) # register interest in suites s1,s2,s3 and any new suites ci.ch_auto_add( False ) # disable adding newly created suites to my handle except RuntimeError, e: print(str(e))
ch_auto_add( (Client)arg1, (bool)arg2) -> int
ch_drop((Client)arg1, (int)arg2) → int :
Drop/de-register the client handle.
When dealing with large definitions, where a user is only interested in a small subset of suites, registering them, improves download performance from the server. Registered suites have an associated handle. Client must ensure un-used handle are dropped otherwise they will stay, in the ecflow_server
void ch_drop( int client_handle : The handle must be an integer that is > 0 ) void ch_drop() : Uses the local handle stored on the client, from last call to ch_register()
Exception:
- RunTimeError thrown if handle has not been previously registered
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.ch_register(False,["s1","s2"]) while( 1 ): # get incremental changes to suites s1 & s2, uses data stored on ci/defs ci.sync_local() # will only retrieve data for suites s1 & s2 update(ci.get_defs()) finally: ci.ch_drop()ch_drop( (Client)arg1) -> int
ch_drop_user((Client)arg1, (str)arg2) → int :
Drop/de-register all handles associated with user.
When dealing with large definitions, where a user is only interested in a small subset of suites, registering them, improves download performance from the server. Registered suites have an associated handle. Client must ensure un-used handle are dropped otherwise they will stay, in the ecflow_server
void ch_drop_user( string user # If empty string will drop current user )Exception:
- RunTimeError thrown if handle has not been previously registered
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.ch_register(False,["s1","s2"]) while( 1 ): # get incremental changes to suites s1 & s2, uses data stored on ci/defs update(ci.get_defs()) finally: ci.ch_drop_user("") # drop all handles associated with current user
ch_handle((Client)arg1) → int :
Register interest in a set of suite s.
If a definition has lots of suites, but the client is only interested in a small subset. Then using this command can reduce network bandwidth and synchronisation will be quicker. This command will create a client handle. This handle is held locally on the
ecflow.Client, and can be used implicitly by ch_drop(),ch_add(),ch_remove() and ch_auto_add(). Registering a client handle affects the news() and sync() commands:void ch_register( bool auto_add_new_suites : true means add new suites to my list, when they are created list suite_names : should be a list of suite names, names not in the definition are ignored )
Usage:
try: ci = Client() suite_names = [ "s1", "s2", "s3" ] ci.ch_register(True,suite_names) # register interest in suites s1,s2,s3 and any new suites ci.ch_register(False,suite_names) # register interest in suites s1,s2,s3 only except RuntimeError, e: print(str(e))The client ‘ci’ will hold locally the client handle. Since we have made multiple calls to register a handle, the variable ‘ci’ will hold the handle for the last call only. The handle associated with the suite can be manually retrieved:
try: ci = Client() ci.ch_register(True,["s1","s2","s3"]) # register interest in suites s1,s2,s3 and any new suites client_handle = ci.ch_handle() # get the handle associated with last call to ch_register .... # after a period of time except RuntimeError, e: print(str(e)) finally: ci.ch_drop( client_handle ) # de-register the handle
ch_register((Client)arg1, (bool)arg2, (list)arg3) → None :
Register interest in a set of suite s.
If a definition has lots of suites, but the client is only interested in a small subset. Then using this command can reduce network bandwidth and synchronisation will be quicker. This command will create a client handle. This handle is held locally on the
ecflow.Client, and can be used implicitly by ch_drop(),ch_add(),ch_remove() and ch_auto_add(). Registering a client handle affects the news() and sync() commands:void ch_register( bool auto_add_new_suites : true means add new suites to my list, when they are created list suite_names : should be a list of suite names, names not in the definition are ignored )
Usage:
try: ci = Client() suite_names = [ "s1", "s2", "s3" ] ci.ch_register(True,suite_names) # register interest in suites s1,s2,s3 and any new suites ci.ch_register(False,suite_names) # register interest in suites s1,s2,s3 only except RuntimeError, e: print(str(e))The client ‘ci’ will hold locally the client handle. Since we have made multiple calls to register a handle, the variable ‘ci’ will hold the handle for the last call only. The handle associated with the suite can be manually retrieved:
try: ci = Client() ci.ch_register(True,["s1","s2","s3"]) # register interest in suites s1,s2,s3 and any new suites client_handle = ci.ch_handle() # get the handle associated with last call to ch_register .... # after a period of time except RuntimeError, e: print(str(e)) finally: ci.ch_drop( client_handle ) # de-register the handle
ch_remove((Client)arg1, (int)arg2, (list)arg3) → None :
- Remove a set of suites, from an existing handle
When dealing with large definitions, where a user is only interested in a small subset of suites, registering them, improves download performance from the server. Registered suites have an associated handle.
integer ch_remove( integer handle : the handle obtained after ch_register list suite_names : list of strings representing suite names ) integer ch_remove( list suite_names : list of strings representing suite names )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.ch_register(True,["s1","s2","s3"]) # register interest in suites s1,s2,s3 and any new suites ci.ch_remove( ["s1"] ) # remove suites s1 from the last added handle except RuntimeError, e: print(str(e))
ch_remove( (Client)arg1, (list)arg2) -> None
ch_suites((Client)arg1) → None :
Writes to standard out the list of registered handles and the suites they reference.
When dealing with large definitions, where a user is only interested in a small subset of suites, registering them, improves download performance from the server. Registered suites have an associated handle.
check((Client)arg1, (str)arg2) → str :
Check trigger and complete expression s and limit s
The ecflow_server does not store extern s. Hence all unresolved references are reported as errors. Returns a non empty string for any errors or warning
string check( list paths # List of paths. ) string check( string absolute_node_path )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) print(ci.check("/suite1")) except RuntimeError, e: print(str(e))check( (Client)arg1, (list)arg2) -> str
checkpt((Client)arg1[, (CheckPt)mode=ecflow.CheckPt.UNDEFINED[, (int)check_pt_interval=0[, (int)check_pt_save_alarm_time=0]]]) → int :
Request the ecflow_server check point s the definition held in the server immediately
This effectively saves the definition held in the server to disk, in a platform independent manner. This is the default when no arguments are specified. The saved file will include node state, passwords, etc. The default file name is <host>.<port>.ecf.check and is saved in ECF_HOME directory. The check point file name can be overridden via ECF_CHECK server environment variable. The back up check point file name can be overridden via ECF_CHECKOLD server environment variable:
void checkpt( [(CheckPt::Mode)mode=CheckPt.UNDEFINED] : Must be one of [ NEVER, ON_TIME, ALWAYS, UNDEFINED ] NEVER : Never check point the definition in the server ON_TIME: Turn on automatic check pointing at interval stored on server or with interval specified as the second argument ALWAYS: Check point at any change in node tree, *NOT* recommended for large definitions UNDEFINED:The default, which allows for immediate check pointing, or alarm setting [(int)interval=120] : This specifies the interval in seconds when server should automatically check pt. This will only take effect if mode is on_time/CHECK_ON_TIME Should ideally be a value greater than 60 seconds, default is 120 seconds [(int)alarm=30] : Specifies check pt save alarm time. If saving the check pt takes longer than the alarm time, then the late flag is set on the server. This flag will need to be cleared manually. )Note
When the time taken to save the check pt is excessive, it can interfere with job scheduling. It may be an indication of the following:
- slow disk
- file system full
- The definition is very large and needs to split
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.checkpt() # Save the definition held in the server to disk ci.checkpt(CheckPt.NEVER) # Switch off check pointing ci.checkpt(CheckPt.ON_TIME) # Start automatic check pointing at the interval stored in the server ci.checkpt(CheckPt.ON_TIME,180) # Start automatic check pointing every 180 seconds ci.checkpt(CheckPt.ALWAYS) # Check point at any state change in node tree. *not* recommended for large defs ci.checkpt(CheckPt.UNDEFINED,0,35) # Change check point save time alarm to 35 seconds # With these arguments mode and interval remain unchanged except RuntimeError, e: print(str(e))
child_abort((Client)arg1[, (str)reason='']) → None :
Child command,notify server job has aborted, can provide an optional reason
child_complete((Client)arg1) → None :
Child command,notify server job has complete
child_event((Client)arg1, (str)arg2) → None :
Child command,notify server event occurred, requires the event name
child_init((Client)arg1) → None :
Child command,notify server job has started
child_label((Client)arg1, (str)arg2, (str)arg3) → None :
Child command,notify server label changed, requires label name, and new value
child_meter((Client)arg1, (str)arg2, (int)arg3) → None :
Child command,notify server meter changed, requires meter name and value
child_wait((Client)arg1, (str)arg2) → None :
Child command,wait for expression to come true
clear_log((Client)arg1) → int :
Request the ecflow_server to clear log file. Log file will be empty after this call.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.clear_log() # log file is now empty except RuntimeError, e: print(str(e))
debug_server_off((Client)arg1) → int :
Disable server debug
debug_server_on((Client)arg1) → int :
Enable server debug, Will dump to standard out on server host.
delete((Client)arg1, (str)abs_node_path[, (bool)force=False]) → int :
Delete the node (s) specified.
If a node is submitted or active, then a Exception will be raised. To force the deletion at the expense of zombie creation, then set the force parameter to true
void delete( list paths : List of paths. [(bool)force=False] : If true delete even if in "active" or "submitted" states Which risks creating zombies. ) void delete( string absolute_node_path: Path name of node to delete. [(bool)force=False] : If true delete even if in "active" or "submitted" states )Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.delete("/s1/f1/task1") paths = ["/s1/f1/t1","/s2/f1/t2"] ci.delete(paths) # delete all tasks specified in the paths except RuntimeError, e: print(str(e))delete( (Client)arg1, (list)paths [, (bool)force=False]) -> None
delete_all((Client)arg1[, (bool)force=False]) → int :
Delete all the node s held in the ecflow_server.
The suite definition in the server will be empty, after this call. Use with care If a node is submitted or active, then a Exception will be raised. To force the deletion at the expense of zombie creation, then set the force parameter to true
void delete_all( [(bool)force=False] : If true delete even if in "active" or "submitted" states Which risks creating zombies. )Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.delete_all() ci.get_server_defs() except RuntimeError, e: print(str(e)); # expect failure since all nodes deleted
flush_log((Client)arg1) → int :
Request the ecflow_server to flush and then close log file
It is best that the server is shutdown first, as log file will be reopened whenever a command wishes to log any changes.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.flush_log() # Log can now opened by external program except RuntimeError, e: print(str(e))
force_event((Client)arg1, (str)arg2, (str)arg3) → None :
- Set or clear a event
void force_event( string absolute_node_path:event: Path name to node: < event name | number> The paths must begin with a leading "/" string signal : [ set | clear ] ) void force_event( list paths : A list of absolute node paths. Each path must include a event name The paths must begin with a leading "/" string signal : [ set | clear ] )Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.force_event("/s1/f1:event_name","set") # Set or clear a event for a list of events paths = [ "/s1/t1:ev1", "/s2/t2:ev2" ] ci.force_event(paths,"clear") except RuntimeError, e: print(str(e))
force_event( (Client)arg1, (list)arg2, (str)arg3) -> None
force_state((Client)arg1, (str)arg2, (State)arg3) → None :
Force a node(s) to a given state
When a task is set to complete, it may be automatically re-queued if it has multiple time dependencies. In the specific case where a task has a single time dependency and we want to interactively set it to complete a flag is set so that it is not automatically re-queued when set to complete. The flag is applied up the node hierarchy until reach a node with a repeat or cron attribute. This behaviour allow repeat values to be incremented interactively. A repeat attribute is incremented when all the child nodes are complete in this case the child nodes are automatically re-queued
void force_state( string absolute_node_path: Path name to node. The path must begin with a leading "/" State::State state : [ unknown | complete | queued | submitted | active | aborted ] ) void force_state( list paths : A list of absolute node paths. The paths must begin with a leading "/" State::State state : [ unknown | complete | queued | submitted | active | aborted ] )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) # force a single node to complete ci.force_state("/s1/f1",State.complete) # force a list of nodes to complete paths = [ "/s1/t1", "/s1/t2", "/s1/f1/t1" ] ci.force_state(paths,State.complete) except RuntimeError, e: print(str(e))Effect:
Lets see the effect of forcing complete on the following defs
suite s1 task t1; time 10:00 # will complete straight away task t2; time 10:00 13:00 01:00 # will re-queue 3 times and complete on fourth
In the last case (task t2) after each force complete, the next time slot is incremented. This can be seen by calling the Why command.
force_state( (Client)arg1, (list)arg2, (State)arg3) -> None
force_state_recursive((Client)arg1, (str)arg2, (State)arg3) → None :
- Force node(s) to a given state recursively
void force_state_recursive( string absolute_node_path: Path name to node.The paths must begin with a leading "/" State::State state : [ unknown | complete | queued | submitted | active | aborted ] ) void force_state_recursive( list paths : A list of absolute node paths.The paths must begin with a leading "/" State::State state : [ unknown | complete | queued | submitted | active | aborted ] )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.force_state_recursive("/s1/f1",State.complete) # recursively force a list of nodes to complete paths = [ "/s1", "/s2", "/s1/f1/t1" ] ci.force_state_recursive(paths,State.complete) except RuntimeError, e: print(str(e))
force_state_recursive( (Client)arg1, (list)arg2, (State)arg3) -> None
free_all_dep((Client)arg1, (str)arg2) → None :
- Free all trigger, date and all time(day, today, cron,etc) dependencies
void free_all_dep( string absolute_node_path : Path name to node )
After freeing the time related dependencies (i.e.. time,today,cron) the next time slot will be missed.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.free_all_dep("/s1/task") except RuntimeError, e: print(str(e))
free_all_dep( (Client)arg1, (list)arg2) -> None
free_date_dep((Client)arg1, (str)arg2) → None :
- Free date dependencies for a node
void free_date_dep( string absolute_node_path : Path name to node )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.free_date_dep("/s1/task") except RuntimeError, e: print(str(e))
free_date_dep( (Client)arg1, (list)arg2) -> None
free_time_dep((Client)arg1, (str)arg2) → None :
- Free all time dependencies. i.e.. time, day, today, cron
void free_time_dep( string absolute_node_path : Path name to node )
After freeing the time related dependencies (i.e. time,today,cron) the next time slot will be missed.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.free_time_dep("/s1/task") except RuntimeError, e: print(str(e))
free_time_dep( (Client)arg1, (list)arg2) -> None
free_trigger_dep((Client)arg1, (str)arg2) → None :
- Free trigger dependencies for a node
void free_trigger_dep( string absolute_node_path : Path name to node )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.free_trigger_dep("/s1/f1/task") except RuntimeError, e: print(str(e))
free_trigger_dep( (Client)arg1, (list)arg2) -> None
get_defs((Client)arg1) → Defs :
Returns the suite definition stored on the Client.
Use
ecflow.Client.sync_local()to retrieve the definition from the server first. The definition is retained in memory until the next call to sync_local().Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.sync_local() # get the definition from the server and store on "ci" print(ci.get_defs()) # print out definition stored in the client print(ci.get_defs()) # print again, this shows that defs is retained on ci except RuntimeError, e: print(str(e))
get_file((Client)arg1, (str)arg2, (str)arg3, (str)arg4) → str :
File command can be used to request the various file types associated with a node
This command defaults to returning a max of 10000 lines. This can be changed
string get_file( string absolute_node_path : Path name to node [(string)file_type="script"] : file_type = [ script<default> | job | jobout | manual | kill | stat ] [(string)max_lines="10000"] : The number of lines in the file to return )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) for file in [ "script", "job", "jobout", "manual", "kill", "stat" ]: print(ci.get_file("/suite/f1/t1",file)) # print the contents of the file except RuntimeError, e: print(str(e))get_file( (Client)arg1, (str)arg2, (str)arg3) -> str
get_host((Client)arg1) → str :
Return the host, assume set_host_port() has been set, otherwise return localhost
get_port((Client)arg1) → str :
Return the port, assume set_host_port() has been set. otherwise returns 3141
get_server_defs((Client)arg1) → int :
Get all suite Node tree’s from the ecflow_server.
The definition is retained in memory until the next call to get_server_defs(). This is important since get_server_defs() could return several megabytes of data. Hence we only want to call it once, and then access it locally with get_defs(). If you need to access the server definition in a loop use
ecflow.Client.sync_localinstead since this is capable of returning incremental changes, and thus considerably reducing the network load.Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.get_server_defs() # get the definition from the server and store on "ci" print(ci.get_defs()) # print out definition stored in the client print(ci.get_defs()) # print again, this shows that defs is retained on ci except RuntimeError, e: print(str(e))
group((Client)arg1, (str)arg2) → int :
Allows a series of commands to be executed in the ecflow_server
void group( string cmds : a list of ";" separated commands )Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.group("get; show") ci.group("get; show state") # show node states and trigger abstract syntax trees except RuntimeError, e: print(str(e))
halt_server((Client)arg1) → int :
Halt the ecflow_server
Stop server communication with jobs, and new job scheduling, and stops check pointing. See server states
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.halt_server() except RuntimeError, e: print(str(e))
in_sync((Client)arg1) → bool :
Returns true if the definition on the client is in sync with the ecflow_server
Warning
Calling in_sync() is only valid after a call to sync_local().
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.sync_local() # very first call gets the full Defs client_defs = ci.get_defs() # End user access to the returned Defs ... after a period of time ci.sync_local() # Subsequent calls to sync_local() users the local Defs to sync incrementally if ci.in_sync(): # returns true changed and changes applied to client print("Client is now in sync with server") client_defs = ci.get_defs() # End user access to the returned Defs except RuntimeError, e: print(str(e))
job_generation((Client)arg1, (str)arg2) → int :
Job submission for chosen Node based on dependencies The ecflow_server traverses the node tree every 60 seconds, and if the dependencies are free does job creation and submission. Sometimes the user may free time/date dependencies to avoid waiting for the server poll, this commands allows early job generation
void job_generation( string absolute_node_path: Path name for job generation to start from ) If empty string specified generates for full definition.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.job_generation("/s1") # generate jobs for suite "/s1 except RuntimeError, e: print(str(e))
kill((Client)arg1, (str)arg2) → None :
- Kills the job associated with the node
void kill( list paths: List of paths. Paths must begin with a leading "/" character ) void kill( string absolute_node_path: Path name to node to kill. )
If a family or suite is selected, will kill hierarchically. Kill uses the ECF_KILL_CMD variable. After variable substitution it is invoked as a command. The ECF_KILL_CMD variable should be written in such a way that the output is written to %ECF_JOB%.kill, i.e..:
kill -15 %ECF_RID% > %ECF_JOB%.kill 2>&1 /home/ma/emos/bin/ecfkill %USER% %HOST% %ECF_RID% %ECF_JOB% > %ECF_JOB%.kill 2>&1
Exceptions can be raised because:
- The absolute_node_path does not exist in the server
- ECF_KILL_CMD variable is not defined
- variable substitution fails
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.kill("/s1/f1") time.sleep(2) print(ci.file("/s1/t1","kill")) # request kill output except RuntimeError, e: print(str(e))
kill( (Client)arg1, (list)arg2) -> None
load((Client)arg1, (str)path_to_defs[, (bool)force=False[, (bool)check_only=False[, (bool)print=False]]]) → int :
- Load a suite definition or checkpoint file given by the file_path argument into the ecflow_server
void load( string file_path : path name to the definition file [(bool)force=False] : If true overwrite suite of same name [(bool)print=False] : print parsed defs to standard out )
By default throws a RuntimeError exception for errors. If force is not used and suite of the same name already exists in the server, then a error is thrown
Usage:
defs_file = "Hello.def" defs = Defs() suite = def.add_suite("s1") family = suite.add_family("f1") for i in [ "_1", "_2", "_3" ]: family.add_task( "t" + i ) defs.save_as_defs(defs_file) # write out in memory defs into the file "Hello.def" ... try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.load(defs_file) # open and parse defs or checkpoint file, and load into server. except RuntimeError, e: print(str(e))- load( (Client)arg1, (Defs)defs [, (bool)force=False]) -> int :
Load a in memory suite definition into the ecflow_server
void load( Defs defs : A in memory definition [(bool)force=False] : for true overwrite suite of same name )
If force is not used and suite already exists in the server, then a error is thrown.
Usage:
defs = Defs() suite = defs.add_suite("s1") family = suite.add_family("f1") for i in [ "_1", "_2", "_3" ]: family.add_task( Task( "t" + i) ) ... try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.load(defs) # Load in memory defs, into the server except RuntimeError, e: print(str(e))
log_msg((Client)arg1, (str)arg2) → int :
Request the ecflow_server writes a string message to the log file.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.log_msg("A message") # Write message to log file except RuntimeError, e: print(str(e))
new_log((Client)arg1[, (str)path='']) → int :
Request the ecflow_server to use the path provided, as the new log file
The old log file is released.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.new_log("/path/log.log") # use "/path/log,log" as the new log file # To keep track of log file Can change ECF_LOG ci.alter("","change","variable","ECF_LOG","/new/path.log") ci.new_log() except RuntimeError, e: print(str(e))
news_local((Client)arg1) → bool :
Query the ecflow_server to detect any changes.
This returns a simple bool, if there has been changes, the user should call
ecflow.Client.sync_local. This will bring the client in sync with changes in the server. If sync_local() is not called then calling news_local() will always return true. news_local() uses the definition stored on the client:bool news_local()
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) if ci.news_local(): # has the server changed print("Server Changed") # server changed bring client in sync with server ci.sync_local() # get the full definition from the server if first time # otherwise apply incremental changes to Client definition, # bringing it in sync with the server definition print(ci.get_defs()) # print the synchronised definition. Should be same as server except RuntimeError, e: print(str(e))
order((Client)arg1, (str)arg2, (str)arg3) → None :
Re-orders the node s in the suite definition held by the ecflow_server
It should be noted that in the absence of dependencies, the order in which task s are submitted, depends on the order in the definition. This changes the order and hence affects the submission order
void order( string absolute_node_path: Path name to node. string order_type : Must be one of [ top | bottom | alpha | order | up | down ] ) o top raises the node within its parent, so that it is first o bottom lowers the node within its parent, so that it is last o alpha Arranges for all the peers of selected note to be sorted alphabetically o order Arranges for all the peers of selected note to be sorted in reverse alphabet o up Moves the selected node up one place amongst its peers o down Moves the selected node down one place amongst its peers
Exceptions can be raised because:
- The absolute_node_path does not exist in the server
- The order_type is not the right type
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.order("/s1/f1","top") except RuntimeError, e: print(str(e))
ping((Client)arg1) → int :
Checks if the ecflow_server is running
void ping()
The default behaviour is to check on host ‘localhost’ and port 3141 It should be noted that any Client function will fail if the server is is not running. Hence ping() is not strictly required. However its main distinction from other Client function is that it is quite fast.
Usage:
try: ci = Client("localhost","3150") ci.ping() print("------- Server already running------") do_something_with_server(ci) except RuntimeError, e: print("------- Server *NOT* running------" + str(e))
plug((Client)arg1, (str)arg2, (str)arg3) → int :
Plug command is used to move node s
The destination node can be on another ecflow_server. In which case the destination path should be of the form ‘//<host>:<port>/suite/family/task
void plug( string source_absolute_node_path : Path name to source node string destination_absolute_node_path : Path name to destination node. Note if only "//host:port" is specified the whole suite can be moved )By default throws a RuntimeError exception for errors.
Exceptions can be raised because:
- Source node is in a active or submitted state.
- Another user already has an lock.
- source/destination paths do not exist on the corresponding servers
- If the destination node path is empty, i.e... only host:port is specified, then the source node must correspond to a suite.
- If the source node is added as a child, then its name must be unique
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.plug("/suite","host3:3141") except RuntimeError, e: print(str(e))
reload_wl_file((Client)arg1) → int :
Request that the ecflow_server reload the white list file.
The white list file if present, can be used to control who has read/write access to the ecflow_server:
void reload_wl_file()
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.reload_wl_file() except RuntimeError, e: print(str(e))
replace((Client)arg1, (str)arg2, (str)arg3, (bool)arg4, (bool)arg5) → int :
- Replaces a node in a suite definition with the given path. The definition is in the ecflow_server
void replace( string absolute_node_path: Path name to node in the client defs. This is also the node we want to replace in the server. string client_defs_file : File path to defs files, that provides the definition of the new node [(bool)parent=False] : create parent families or suite as needed, when absolute_node_path does not exist in the server [(bool)force=False] : check for zombies, if force = true, bypass checks ) void replace( string absolute_node_path: Path name to node in the client defs. This is also the node we want to replace in the server. Defs client_defs : In memory client definition that provides the definition of the new node [(bool)parent=False] : create parent families or suite as needed, when absolute_node_path does not exist in the server [(bool)force=False] : check for zombies, force = true, bypass checks )Exceptions can be raised because:
- The absolute_node_path does not exist in the provided definition
- The provided client definition must be free of errors
- If the third argument is not provided, then the absolute_node_path must exist in the server defs
- replace will fail, if child task nodes are in active / submitted state
After replace is done, we check trigger expressions. These are reported to standard output. It is up to the user to correct invalid trigger expressions, otherwise the tasks will not run. Please note, you can use check() to check trigger expression and limits in the server.
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.replace("/s1/f1","/tmp/defs.def") except RuntimeError, e: print(str(e)) try: ci.replace("/s1",client_defs) # replace suite "s1" in the server, with "s1" in the client_defs except RuntimeError, e: print(str(e))
replace( (Client)arg1, (str)arg2, (Defs)arg3, (bool)arg4, (bool)arg5) -> int
replace( (Client)arg1, (str)arg2, (Defs)arg3) -> None
replace( (Client)arg1, (str)arg2, (str)arg3) -> None
requeue((Client)arg1, (str)abs_node_path[, (str)option='']) → None :
- Re queues the specified node (s)
void requeue( list paths : A list of paths. Node paths must begin with a leading "/" character [(str)option=""] : option = ("" | "abort" | "force") "" : empty string, the default, re-queue the node abort: means re-queue only aborted tasks below node force: means re-queueing even if there are nodes that are active or submitted ) void requeue( string absolute_node_path : Path name to node [(string)option=""] : option = ("" | "abort" | "force") )Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.requeue("/s1","abort") # re-queue aborted tasks below suite /s1 path_list = ["/s1/f1/t1","/s2/f1/t2"] ci.requeue(path_list) except RuntimeError, e: print(str(e))
requeue( (Client)arg1, (list)paths [, (str)option=’‘]) -> None
reset((Client)arg1) → None :
reset client definition, and handle number
restart_server((Client)arg1) → int :
Restart the ecflow_server
Start job scheduling, communication with jobs, and respond to all requests. See server states
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.retstart_server() except RuntimeError, e: print(str(e))
restore_from_checkpt((Client)arg1) → int :
Request the ecflow_server loads the check point file from disk
The server will first try to open file at ECF_HOME/ECF_CHECK if that fails it will then try path ECF_HOME/ECF_CHECKOLD. An error is returned if the server has not been halted or contains a suite definition
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.halt_server() # server must be halted, otherwise restore_from_checkpt will throw ci.restore_from_checkpt() # restore the definition from the check point file except RuntimeError, e: print(str(e))
resume((Client)arg1, (str)arg2) → None :
- Resume job creation / generation for the given node
void resume( list paths: List of paths. Paths must begin with a leading "/" character ) void resume( string absolute_node_path: Path name to node to resume. )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.resume("/s1/f1/task1") paths = ["/s1/f1/t1","/s2/f1/t2"] ci.resume(paths) except RuntimeError, e: print(str(e))
resume( (Client)arg1, (list)arg2) -> None
run((Client)arg1, (str)arg2, (bool)arg3) → None :
Immediately run the jobs associated with the input node.
Ignore trigger s, limit s, suspended, time or date dependencies, just run the task. When a job completes, it may be automatically re-queued if it has multiple time dependencies. In the specific case where a task has a SINGLE time dependency and we want to avoid re running the task then a flag is set so that it is not automatically re-queued when set to complete. The flag is applied up the node hierarchy until we reach a node with a repeat or cron attribute. This behaviour allow repeat values to be incremented interactively. A repeat attribute is incremented when all the child nodes are complete in this case the child nodes are automatically re-queued
void run( string absolute_node_path : Path name to node. If the path is suite/family will recursively run all child tasks [(bool)force=False] : If true, run even if there are nodes that are active or submitted. ) void run( list paths : List of paths. If the path is suite/family will recursively run all child tasks [(bool)force=False] : If true, run even if there are nodes that are active or submitted. )Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.run("/s1") # run all tasks under suite /s1 ci.run(["/s1/f1/t1","/s2/f1/t2"]) # run all tasks specified except RuntimeError, e: print(str(e))Effect:
Lets see the effect of run command on the following defs:: suite s1 task t1; time 10:00 # will complete straight away task t2; time 10:00 13:00 01:00 # will re-queue 3 times and complete on fourth run
In the last case (task t2) after each run the next time slot is incremented. This can be seen by calling the Why command.
run( (Client)arg1, (list)arg2, (bool)arg3) -> None
server_version((Client)arg1) → str :
Returns the server version, can throw for old servers, that did not implement this request.
set_child_password((Client)arg1, (str)arg2) → None :
Set the password, needed for authentication, provided by the server using %ECF_PASS% By default the environment variable ECF_PASS is read for the jobs password This can be overridden for the python child api
set_child_path((Client)arg1, (str)arg2) → None :
Set the path to the task, obtained from server using %ECF_NAME% By default the environment variable ECF_NAME is read for the task path This can be overridden for the python child api
set_child_pid((Client)arg1, (str)arg2) → None :
- Set the process id of this job
- By default the environment variable ECF_RID is read for the jobs process or remote id This can be overridden for the python child api
- set_child_pid( (Client)arg1, (int)arg2) -> None :
- Set the process id of this job By default the environment variable ECF_RID is read for the jobs process or remote id This can be overridden for the python child api
set_child_timeout((Client)arg1, (int)arg2) → None :
Set timeout if child can not connect to server, default is 24 hours. The input is required to be in seconds By default the environment variable ECF_TIMEOUT is read to control how long child command should attempt to connect to the server This can be overridden for the python child api
set_child_try_no((Client)arg1, (int)arg2) → None :
Set the try no, i.e.. the number of times this job has run, obtained from the server, using %ECF_TRYNO% By default the environment variable ECF_TRYNO is read to record number of times job has been run This can be overridden for the python child api
set_connection_attempts((Client)arg1, (int)arg2) → None :
Set the number of times to connect to ecflow_server, in case of connection failures
The period between connection attempts is handled by Client.set_retry_connection_period(). If the network is unreliable the connection attempts can be be increased, likewise when the network is stable this number could be reduced to one. This can increase responsiveness and reduce latency. Default value is set as 2. Setting a value less than one is ignored, will default to 1 in this case:
set_connection_attempts( int attempts # must be an integer >= 1 )
Exceptions:
- None
Usage:
ci = Client() ci.set_connection_attempts(3) # make 3 attempts for server connection ci.set_retry_connection_period(1) # wait 1 second between each attempt
set_host_port((Client)arg1, (str)arg2, (str)arg3) → None :
- Override the default(localhost and port 3141) and environment setting(ECF_HOST and ECF_PORT)
and set it explicitly:
set_host_port( string host, # The server name. Can not be empty. string port # The port on the server, must be unique to the server ) set_host_port( string host, # The server name. Can not be empty. int port # The port on the server, must be unique to the server ) set_host_port( string host_port, # Expect"s <host>:<port> or <host>@<port> )
Exceptions:
- Raise a RuntimeError if the host or port is empty
Usage:
try: ci = Client() ci.set_host_port("localhost","3150") ci.set_host_port("avi",3150) ci.set_host_port("avi:3150") except RuntimeError, e: print(str(e))
set_host_port( (Client)arg1, (str)arg2) -> None
set_host_port( (Client)arg1, (str)arg2, (int)arg3) -> None
set_retry_connection_period((Client)arg1, (int)arg2) → None :
Set the sleep period between connection attempts
Whenever there is a connection failure we wait a number of seconds before trying again. i.e... to get round glitches in the network. For the ping command this is hard wired as 1 second. This wait between connection attempts can be configured here. i.e.. This could be reduced to increase responsiveness. Default: In debug this period is 1 second and in release mode 10 seconds:
set_retry_connection_period( int period # must be an integer >= 0 )
Exceptions:
- None
Usage:
ci = Client() ci.set_connection_attempts(3) # make 3 attempts for server connection ci.set_retry_connection_period(1) # wait 1 second between each attempt
shutdown_server((Client)arg1) → int :
Shut down the ecflow_server
Stop server from scheduling new jobs. See server states
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.shutdown_server() except RuntimeError, e: print(str(e))
sort_attributes((Client)arg1, (str)abs_node_path, (str)attribute_name[, (bool)recursive=True]) → None
sort_attributes( (Client)arg1, (list)paths, (str)attribute_name [, (bool)recursive=True]) -> None
stats((Client)arg1) → None :
Prints the ecflow_server statistics to standard out
void stats()
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.stats() except RuntimeError, e: print(str(e))
stats_reset((Client)arg1) → None :
Resets the statistical data in the server
void stats_reset()
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.stats_reset() except RuntimeError, e: print(str(e))
status((Client)arg1, (str)arg2) → None :
- Shows the status of a job associated with a task
void status( list paths: List of paths. Paths must begin with a leading "/" character ) void status( string absolute_node_path )
If a family or suite is selected, will invoke status command hierarchically. Status uses the ECF_STATUS_CMD variable. After variable substitution it is invoked as a command. The command should be written in such a way that the output is written to %ECF_JOB%.stat, i.e..:
/home/ma/emos/bin/ecfstatus %USER% %HOST% %ECF_RID% %ECF_JOB% > %ECF_JOB%.stat 2>&1
Exceptions can be raised because:
- The absolute_node_path does not exist in the server
- ECF_STATUS_CMD variable is not defined
- variable substitution fails
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.status("/s1/t1") time.sleep(2) print(ci.file("/s1/t1","stats")) # request status output except RuntimeError, e: print(str(e))
status( (Client)arg1, (list)arg2) -> None
suites((Client)arg1) → list :
Returns a list strings representing the suite names
list(string) suites()
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) suites = ci.suites() print(suites) except RuntimeError, e: print(str(e))
suspend((Client)arg1, (str)arg2) → None :
- Suspend job creation / generation for the given node
void suspend( list paths: List of paths. Paths must begin with a leading "/" character ) void suspend( string absolute_node_path: Path name to node to suspend. )
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.suspend("/s1/f1/task1") paths = ["/s1/f1/t1","/s2/f1/t2"] ci.suspend(paths) except RuntimeError, e: print(str(e))
suspend( (Client)arg1, (list)arg2) -> None
sync_local((Client)arg1) → int :
Requests that ecflow_server returns the full definition or incremental change made and applies them to the client Defs
When there is a very large definition, calling
ecflow.Client.get_server_defseach time can be very expensive both in terms of memory, speed, and network bandwidth. The alternative is to call this function, which will get the incremental changes, and apply them local client suite definition effectively synchronising the client and server Defs. If the period of time between two sync() calls is too long, then the full server definition is returned and assigned to the client Defs. We can determine if the changes were applied by calling in_sync() after the call to sync_local():void sync_local(); # The very first call, will get the full Defs.
Exceptions:
- raise a RuntimeError if the delta change can not be applied.
- this could happen if the client Defs bears no resemblance to server Defs
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.sync_local() # Very first call gets the full Defs client_defs = ci.get_defs() # End user access to the returned Defs ... after a period of time ci.sync_local() # Subsequent calls to sync_local() users the local Defs to sync incrementally if ci.in_sync(): # returns true server changed and changes applied to client print("Client is now in sync with server") client_defs = ci.get_defs() # End user access to the returned Defs except RuntimeError, e: print(str(e))Calling sync_local() is considerably faster than calling get_server_defs() for large Definitions
terminate_server((Client)arg1) → int :
Terminate the ecflow_server
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.terminate_server() except RuntimeError, e: print(str(e))
version((Client)arg1) → str :
Returns the current client version
wait_for_server_reply((Client)arg1[, (int)time_out=60]) → bool :
Wait for a response from the ecflow_server:
void wait_for_server_reply( int time_out : (default = 60) )
This is used to check if server has started. Typically for tests. Returns true if server(ping) replies before time out, otherwise false
Usage:
ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) if ci.wait_for_server_reply(30): print("Server is alive") else: print("Timed out after 30 second wait for server response.?")
zombie_adopt((Client)arg1, (str)arg2) → int
zombie_block((Client)arg1, (str)arg2) → int
zombie_fail((Client)arg1, (str)arg2) → int
zombie_fob((Client)arg1, (str)arg2) → int
zombie_kill((Client)arg1, (str)arg2) → int
zombie_remove((Client)arg1, (str)arg2) → int
changed_node_paths
After a call to sync_local() we can access the list of nodes that changed
The returned list consists of node paths. IF the list is empty assume that
whole definition changed. This should be expected after the first call to sync_local()
since that always retrieves the full definition from the server:
py
Midnight
void changed_node_paths()
Usage:
py
Midnight
try:
ci = Client() # use default host(ECF_HOST) & port(ECF_PORT)
if ci.news_local(): # has the server changed
print("Server Changed") # server changed bring client in sync with server
ci.sync_local() # get the full definition from the server if first time
# otherwise apply incremental changes to Client definition,
# bringing it in sync with the server definition
defs = ci.get_defs() # get the updated/synchronised definition
for path in ci.changed_node_paths:
if path == "/": # path "/" represent change to server node/defs
print("defs changed") # defs state change or user variables changed
else:
node = defs.find_abs_node(path)
# if changed_node_paths is empty, then assume entire definition changed
print(defs) # print the synchronised definition. Should be same as server
except RuntimeError, e:
print(str(e))
class ecflow.Clock
Bases:
Boost.Python.instanceSpecifies the clock type used by the suite.
Only suites can have a clock. A gain can be specified to offset from the given date.
Constructor:
Clock(day,month,year,hybrid) int day : Specifies the day of the month 1-31 int month : Specifies the month 1-12 int year : Specifies the year > 1400 bool hybrid<optional>: Default = False, true means hybrid, false means real by default the clock is not real Time will be set to midnight, use set_gain() to alter Clock(hybrid) bool hybrid: true means hybrid, false means real by default the clock is real Time will be set real time of the computerExceptions:
- raises IndexError when an invalid Clock is specified
Usage:
suite = Suite("s1") clock = Clock(1,1,2012,False) clock.set_gain(1,10,True) suite.add_clock(clock)day((Clock)arg1) → int :
Returns the day as an integer, range 1-31
gain((Clock)arg1) → int :
Returns the gain as an long. This represents seconds
month((Clock)arg1) → int :
Returns the month as an integer, range 1-12
positive_gain((Clock)arg1) → bool :
Returns a boolean, where true means that the gain is positive
set_gain((Clock)arg1, (int)arg2, (int)arg3, (bool)arg4) → None :
Set the gain in hours and minutes
set_gain_in_seconds((Clock)arg1, (int)arg2, (bool)arg3) → None :
Set the gain in seconds
set_virtual((Clock)arg1, (bool)arg2) → None :
Sets/unsets the clock as being virtual
virtual((Clock)arg1) → bool :
Returns a boolean, where true means that clock is virtual
year((Clock)arg1) → int :
Returns the year as an integer, > 1400
class ecflow.Complete
Bases:
Boost.Python.instanceAdd a trigger or complete expression.
This defines a dependency for a node. There can only be one trigger or complete expression dependency per node. A node with a trigger can only be activated when the trigger has expired. Triggers can reference nodes, events, meters, variables, repeats, limits and late flag A trigger holds a node as long as the expression returns false.
Exception:
- Will throw RuntimeError if first expression is added as ‘AND’ or ‘OR’ expression Like wise second and subsequent expression must have ‘AND’ or ‘OR’ booleans set
Usage:
Multiple trigger will automatically be anded together, If or is required please use bool ‘False’ as the last argument i.e..
task1.add( Trigger("t2 == active" ), Trigger("t1 == complete or t4 == complete" ), Trigger("t5 == active",False))The trigger for task1 is equivalent to
t2 == active and t1 == complete or t4 == complete or t5 == active
Since a large number of triggers are of the form <node> == complete there are are short cuts, these involves a use of a list
task1.add( Trigger( ["t2","t3"] )) # This is same as t2 == complete and t3 == complete
You can also use a node
task1.add( Trigger( ["t2",taskx] ))
If the node ‘taskx’ has a parent, we use the full hierarchy, hence we will get a trigger of the form
t2 ==complete and /suite/family/taskx == complete
If however node taskx has not yet been added to its parent, we use a relative name in the trigger
t2 ==complete and taskx == complete
get_expression((Complete)arg1) → str :
returns the complete expression as a string
class ecflow.Cron
Bases:
Boost.Python.instancecron defines a repeating time dependency for a node.
Crons are repeated indefinitely.
Avoid having a cron and repeat at the same level,as both provide looping functionality
Constructor:
Cron() Cron(string time_series, days_of_week=list of ints, # 0-6, Sunday-Saturday days_of_month=list of ints, # 1-31 months=list of ints) # 1-12 Cron(TimeSeries time_series, days_of_week=list of ints, days_of_month=list of ints, months=list of ints)Exceptions:
- raises IndexError || RuntimeError when an invalid cron is specified
Usage:
cron = Cron("+00:00 23:00 00:30", days_of_week=[0,1,2,3,4,5,6],days_of_month=[1,2,3,4,5,6], months=[1,2,3,4,5,6]) # Here "+" means relative to begin or re-queue time cron = Cron("+01:30",days_of_week=[0,1,2,3,4,5,6]) cron = Cron("+00:00 23:00 00:30", days_of_week=[0,1,2],days_of_month=[4,5,6], months=[1,2,3]) start = TimeSlot(0 , 0) finish = TimeSlot(23, 0) incr = TimeSlot(0, 30) ts = TimeSeries(start, finish, incr, True) # True means relative to suite start cron = Cron(ts, days_of_week=[0,1,2,3,4,5,6],days_of_month=[1,2,3,4,5,6], months=[1,2]) cron = Cron() cron.set_week_days([0, 1, 2, 3, 4, 5, 6]) cron.set_days_of_month([1, 2, 3, 4, 5, 6 ]) cron.set_months([1, 2, 3, 4, 5, 6]) cron.set_time_series(ts) cron1 = Cron() cron1.set_time_series(1, 30, True) # same as cron +01:30 cron2 = Cron() cron2.set_week_days([0, 1, 2, 3, 4, 5, 6]) cron2.set_time_series("00:30 01:30 00:01") cron3 = Cron() cron3.set_week_days([0, 1, 2, 3, 4, 5, 6]) cron3.set_time_series("+00:30")set_days_of_month((Cron)arg1, (list)arg2) → None :
Specifies days of the month. Expects a list of integers with integer range 1-31
set_months((Cron)arg1, (list)arg2) → None :
Specifies months. Expects a list of integers, with integer range 1-12
set_time_series((Cron)arg1, (int)hour, (int)minute[, (bool)relative=False]) → None :
time_series(hour(int),minute(int),relative to suite start(bool=false)), Add a time slot
- set_time_series( (Cron)arg1, (TimeSeries)arg2) -> None :
- Add a time series. This will never complete
- set_time_series( (Cron)arg1, (TimeSlot)arg2, (TimeSlot)arg3, (TimeSlot)arg4) -> None :
- Add a time series. This will never complete
- set_time_series( (Cron)arg1, (str)arg2) -> None :
- Add a time series. This will never complete
set_week_days((Cron)arg1, (list)arg2) → None :
Specifies days of week. Expects a list of integers, with integer range 0==Sun to 6==Sat
time((Cron)arg1) → TimeSeries :
return cron time as a TimeSeries
days_of_month returns a integer list of days of the month months returns a integer list of months of the year week_days returns a integer list of week days
class ecflow.DState
Bases:
Boost.Python.enumA DState is like a ecflow.State, except for the addition of SUSPENDED
Suspended stops job generation, and hence is an attribute of a Node. DState can be used for setting the default state of node when it is begun or re queued. DState is used for defining defstatus. See
ecflow.Node.add_defstatusandecflow.DefstatusThe default state of a node is queued.Usage:
task = ecflow.Task("t1") task.add_defstatus(ecflow.DState.complete) task = ecflow.Task("t2") task += Defstatus("complete") task = Task("t3", Defstatus("complete")) # create in place
aborted = ecflow.DState.aborted
active = ecflow.DState.active
complete = ecflow.DState.complete
names = {'complete': ecflow.DState.complete, 'unknown': ecflow.DState.unknown, 'aborted': ecflow.DState.aborted, 'submitted': ecflow.DState.submitted, 'suspended': ecflow.DState.suspended, 'active': ecflow.DState.active, 'queued': ecflow.DState.queued}
queued = ecflow.DState.queued
submitted = ecflow.DState.submitted
suspended = ecflow.DState.suspended
unknown = ecflow.DState.unknown
values = {0: ecflow.DState.unknown, 1: ecflow.DState.complete, 2: ecflow.DState.queued, 3: ecflow.DState.aborted, 4: ecflow.DState.submitted, 5: ecflow.DState.active, 6: ecflow.DState.suspended}
class ecflow.Date
Bases:
Boost.Python.instanceUsed to define a date dependency.
There can be multiple Date dependencies for a node. Any of the 3 attributes, i.e... day, month, year can be wild carded using a zero If a hybrid clock is defined on a suite, any node held by a date dependency will be set to complete at the beginning of the suite, without the task ever being dispatched otherwise, the suite would never complete.
Constructor:
Date(string) string : * means wild card Date(day,month,year) int day : represents the day, zero means wild card. day >= 0 & day < 31 int month : represents the month, zero means wild card. month >= 0 & month < 12 int year : represents the year, zero means wild card. year >= 0
Exceptions:
- raises IndexError when an invalid date is specified
Usage:
date = Date(11,12,2010) # represent 11th of December 2010 date = Date(1,0,0); # means the first day of every month of every year t = Task("t1", Date("1.*.*")); # Create Date in place.day((Date)arg1) → int :
Return the day. The range is 0-31, 0 means its wild-carded
month((Date)arg1) → int :
Return the month. The range is 0-12, 0 means its wild-carded
year((Date)arg1) → int :
Return the year, 0 means its wild-carded
class ecflow.Day
Bases:
Boost.Python.instanceDefines a day dependency.
There can be multiple day dependencies. If a hybrid clock is defined on a suite, any node held by a day dependency will be set to complete at the beginning of the suite, without the task ever being dispatched otherwise the suite would never complete.
Constructor:
Day(string) string: "sunday","monday",etc Day(Days) Days day: Is an enumerator with represent the days of the week
Usage:
day1 = Day(Days.sunday) t = Task("t1", Day("tuesday"))day((Day)arg1) → Days :
Return the day as enumerator
class ecflow.Days
Bases:
Boost.Python.enumThis enum is used as argument to a
ecflow.Dayclass.It represents the days of the week
Usage:
day1 = Day(Days.sunday) day2 = Day(Days.monday) t = Task("t1", day1, day2, Day(Days.tuesday))
friday = ecflow.Days.friday
monday = ecflow.Days.monday
names = {'monday': ecflow.Days.monday, 'tuesday': ecflow.Days.tuesday, 'friday': ecflow.Days.friday, 'wednesday': ecflow.Days.wednesday, 'thursday': ecflow.Days.thursday, 'sunday': ecflow.Days.sunday, 'saturday': ecflow.Days.saturday}
saturday = ecflow.Days.saturday
sunday = ecflow.Days.sunday
thursday = ecflow.Days.thursday
tuesday = ecflow.Days.tuesday
values = {0: ecflow.Days.sunday, 1: ecflow.Days.monday, 2: ecflow.Days.tuesday, 3: ecflow.Days.wednesday, 4: ecflow.Days.thursday, 5: ecflow.Days.friday, 6: ecflow.Days.saturday}
wednesday = ecflow.Days.wednesday
class ecflow.Defs
Bases:
Boost.Python.instanceThe Defs class holds the suite definition structure.
It contains all the
ecflow.Suiteand hence acts like the root for suite node tree hierarchy. The definition can be kept as python code, alternatively it can be saved as a flat ASCII definition file. If a definition is read in from disk, it will by default, check the trigger expressions. If however the definition is created in python, then checking should be done explicitly:- Defs(string)
- string - The Defs class take one argument which represents the file name
- Defs(Suite | Edit )
ecflow.Suite- One or more suitesecflow.Edit- specifies user defined server variables
Example:
# Build definition using Constructor approach, This allows indentation, to show the structure # This is a made up example to demonstrate suite construction: defs = Defs( Edit(SLEEP=10,FRED="bill"), # user defined server variables Suite("s1" Clock(1, 1, 2010, False), Autocancel(1, 10, True), Task("t1" Edit({"a":"12", "b":"bb"}, c="v",d="b"), Edit(g="d"), Edit(h=1), Event(1), Event(11,"event"), Meter("meter",0,10,10), Label("label","c"), Trigger("1==1"), Complete("1==1"), Limit("limit",10),Limit("limit2",10), InLimit("limitName","/limit",2), Defstatus(DState.complete), Today(0,30),Today("00:59"),Today("00:00 11:30 00:01"), Time(0,30),Time("00:59"),Time("00:00 11:30 00:01"), Day("sunday"),Day(Days.monday), Date(1,1,0),Date(28,2,1960), Autocancel(3) ), [ Family("f{}".format(i)) for i in range(1,6)])) defs.save_as_defs("filename.def") # save defs into file defs = Defs() # create an empty defs suite = defs.add_suite("s1") family = suite.add_family("f1") for i in [ "_1", "_2", "_3" ]: family.add_task( "t" + i ) defs.save_as_defs("filename.def") # save defs into fileCreate a Defs from an existing file on disk.
defs = Defs("filename.def") # Will open and parse the file and create the Definition print(defs)add()
- object add(tuple args, dict kwds) :
add(..) provides a way to append Nodes and attributes
This is best illustrated with an example:
defs = Defs().add( Suite("s1").add( Clock(1, 1, 2010, False), Autocancel(1, 10, True), Task("t1").add( Edit({"a":"12", "b":"bb"}, c="v",d="b"), Edit(g="d"), Edit(h=1), Event(1), Event(11,"event"), Meter("meter",0,10,10), Label("label","c"), Trigger("1==1"), Complete("1==1"), Limit("limit",10),Limit("limit2",10), InLimit("limitName","/limit",2), Defstatus(DState.complete), Today(0,30),Today("00:59"),Today("00:00 11:30 00:01"), Time(0,30),Time("00:59"),Time("00:00 11:30 00:01"), Day("sunday"),Day(Days.monday), Date(1,1,0),Date(28,2,1960), Autocancel(3) ), [ Family("f{}".format(i)) for i in range(1,6)]))We can also use ‘+=’ with a list here are a few examples:
defs = Defs(); defs += [ Suite("s2"),Edit({ "x1":"y", "aa1":"bb"}, a="v",b="b") ]defs += [ Suite("s{}".format(i)) for i in range(1,6) ]defs = Defs() defs += [ Suite("suite").add( Task("x"), Family("f").add( [ Task("t{}".format(i)) for i in range(1,6)] ), Task("y"), [ Family("f{}".format(i)) for i in range(1,6) ], Edit(a="b"), [ Task("t{}".format(i)) for i in range(1,6) ], )]It is also possible to use ‘+’
defs = Defs() + Suite("s1") defs.s1 += Autocancel(1, 10, True) defs.s1 += Task("t1") + Edit({ "e":1, "f":"bb"}) +\ Event(1) + Event(11,"event") + Meter("meter",0,10,10) + Label("label","c") + Trigger("1==1") +\ Complete("1==1") + Limit("limit",10) + Limit("limit2",10) + InLimit("limitName","/limit",2) +\ Defstatus(DState.complete) + Today(0,30) + Today("00:59") + Today("00:00 11:30 00:01") +\ Time(0,30) + Time("00:59") + Time("00:00 11:30 00:01") + Day("sunday") + Day(Days.monday) +\ Date(1,1,0) + Date(28,2,1960) + Autocancel(3)Warning
We can only use ‘+’ when the left most object is a node, i.e.. Task(‘t1’) in this case
add_extern((Defs)arg1, (str)arg2) → None :
extern refer to nodes that have not yet been defined typically due to cross suite dependencies
trigger and complete expression s may refer to paths, and variables in other suites, that have not been loaded yet. The references to node paths and variable must exist, or exist as externs Externs can be added manually or automatically.
Manual Method:
void add_extern(string nodePath )
Usage:
defs = Defs("file.def") .... defs.add_extern("/temp/bill:event_name") defs.add_extern("/temp/bill:meter_name") defs.add_extern("/temp/bill:repeat_name") defs.add_extern("/temp/bill:edit_name") defs.add_extern("/temp/bill")- Automatic Method:
- This will scan all trigger and complete expressions, looking for paths and variables that have not been defined. The added benefit of this approach is that duplicates will not be added. It is the user’s responsibility to check that extern’s are eventually defined otherwise trigger expression will not evaluate correctly
void auto_add_externs(bool remove_existing_externs_first )
Usage:
defs = Defs("file.def") ... defs.auto_add_externs(True) # remove existing extern first.
add_suite((Defs)arg1, (Suite)arg2) → Suite :
Add a suite node. See
ecflow.SuiteIf a new suite is added which matches the name of an existing suite, then an exception is thrown.
Exception:
- Throws RuntimeError is the suite name is not valid
- Throws RuntimeError if duplicate suite is added
Usage:
defs = Defs() # create a empty defs suite = Suite("suite") # create a stand alone Suite defs.add_suite(suite) # add suite to defs s2 = defs.add_suite("s2") # create a suite and add to defs # Alternatively we can create Suite in place defs = Defs( Suite("s1", Family("f1", Task("t1"))), Suite("s2", Family("f1", Task("t1"))))- add_suite( (Defs)arg1, (str)arg2) -> Suite :
- Create a empty Defs
add_variable((Defs)arg1, (str)arg2, (str)arg3) → Defs :
Adds a name value variable. Also see
ecflow.EditThis defines a variable for use in variable substitution in a ecf script file. There can be any number of variables. The variables are names inside a pair of ‘%’ characters in an ecf script. The name are case sensitive. Special character in the value, must be placed inside single quotes if misinterpretation is to be avoided. The value of the variable replaces the variable name in the ecf script at job creation time. The variable names for any given node must be unique. If duplicates are added then the the last value added is kept.
Exception:
- Writes warning to standard output, if a duplicate variable name is added
Usage:
task.add_variable( Variable("ECF_HOME","/tmp/")) task.add_variable( "TMPDIR","/tmp/") task.add_variable( "COUNT",2) a_dict = { "name":"value", "name2":"value2", "name3":"value3" } task.add_variable(a_dict)add_variable( (Defs)arg1, (str)arg2, (int)arg3) -> Defs
add_variable( (Defs)arg1, (Variable)arg2) -> Defs
add_variable( (Defs)arg1, (dict)arg2) -> Defs
auto_add_externs((Defs)arg1, (bool)arg2) → None :
extern refer to nodes that have not yet been defined typically due to cross suite dependencies
trigger and complete expression s may refer to paths, and variables in other suites, that have not been loaded yet. The references to node paths and variable must exist, or exist as externs Externs can be added manually or automatically.
Manual Method:
void add_extern(string nodePath )
Usage:
defs = Defs("file.def") .... defs.add_extern("/temp/bill:event_name") defs.add_extern("/temp/bill:meter_name") defs.add_extern("/temp/bill:repeat_name") defs.add_extern("/temp/bill:edit_name") defs.add_extern("/temp/bill")- Automatic Method:
- This will scan all trigger and complete expressions, looking for paths and variables that have not been defined. The added benefit of this approach is that duplicates will not be added. It is the user’s responsibility to check that extern’s are eventually defined otherwise trigger expression will not evaluate correctly
void auto_add_externs(bool remove_existing_externs_first )
Usage:
defs = Defs("file.def") ... defs.auto_add_externs(True) # remove existing extern first.
check((Defs)arg1) → str :
Check trigger and complete expression s and limit s
- Client Side: The client side can specify externs. Hence all node path references in trigger expressions, and inlimit references to limit s, that are unresolved and which do not appear in extern s are reported as errors
- Server Side: The server does not store externs. Hence all unresolved references are reported as errors
Returns a non empty string for any errors or warning
Usage:
# Client side defs = Defs("my.def") # Load my.def from disk .... print(defs.check()) # do the check # Server Side try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) print(ci.check("/suite")) except RuntimeError, e: print(str(e))
check_job_creation((Defs)arg1[, (bool)throw_on_error=False[, (bool)verbose=False]]) → str :
Check job creation .
Will check the following:
- ecf script files and includes files can be located
- recursive includes
- manual and comments pre-processing
- variable substitution
Some task s are dummy tasks have no associated ecf script file. To disable error message for these tasks please add a variable called ECF_DUMMY_TASK to them. Checking is done in conjunction with the class
ecflow.JobCreationCtrl. If no node path is set on class JobCreationCtrl then all tasks are checked. In the case where we want to check all tasks, use the convenience function that take no arguments.Usage:
defs = Defs("my.def") # specify the defs we want to check, load into memory ... print(defs.check_job_creation()) # Check job generation for all tasks ... # throw on error and Output the tasks as they are being checked defs.check_job_creation(throw_on_error=TrueTrue,verbose=True) job_ctrl = JobCreationCtrl() defs.set_verbose(True) # Output the tasks as they are being checked defs.check_job_creation(job_ctrl) # Check job generation for all tasks, same as above print(job_ctrl.get_error_msg()) ... job_ctrl = JobCreationCtrl() job_ctrl.set_node_path("/suite/to_check") # will hierarchically check job creation under this node defs.check_job_creation(job_ctrl) # job files generated to ECF_JOB print(job_ctrl.get_error_msg()) ... job_ctrl = JobCreationCtrl() # no set_node_path() hence check job creation for all tasks job_ctrl.set_dir_for_job_creation(tmp) # generate jobs file under this directory defs.check_job_creation(job_ctrl) print(job_ctrl.get_error_msg()) ... job_ctrl = JobCreationCtrl() # no set_node_path() hence check job creation for all tasks job_ctrl.generate_temp_dir() # automatically generate directory for job file defs.check_job_creation(job_ctrl) print(job_ctrl.get_error_msg())check_job_creation( (Defs)arg1, (JobCreationCtrl)arg2) -> None
delete_variable((Defs)arg1, (str)arg2) → None :
An empty string will delete all user variables
find_abs_node((Defs)arg1, (str)arg2) → Node :
Given a path, find the the node
find_suite((Defs)arg1, (str)arg2) → Suite :
Given a name, find the corresponding suite
generate_scripts((Defs)arg1) → None :
Automatically generate template ecf script s for this definition Will automatically add child command s for event, meter and label s. This allows the definition to be refined with out worrying about the scripts. However it should be noted that, this will create a lot of duplicated script contents i.e.. in the absence of event s, meter s and label s, most of generated ecf script files will be the same. Hence should only be used an aid to debugging the definition. It uses the contents of the definition to parameterise what gets generated, and the location of the files. Will throw Exceptions for errors.
Requires:
- ECF_HOME: specified and accessible for all Tasks, otherwise RuntimeError is raised
- ECF_INCLUDE: specifies location for head.h and tail.h includes, will use angle brackets,
- i.e.. %include <head.h>, if the head.h and tail.h already exist they are used otherwise they are generated
Optional:
- ECF_FILES: If specified, then scripts are generated under this directory otherwise ECF_HOME is used.
- The missing directories are automatically created.
- ECF_CLIENT_EXE_PATH: if specified child command will use this, otherwise will use ecflow_client
- and assume this accessible on the path.
- ECF_DUMMY_TASK: Will not generated scripts for this task.
- SLEEP: Uses this variable to delay time between calls to child commands, if not specified uses delay of one second
Usage:
defs = ecflow.Defs() suite = defs.add_suite("s1") suite.add_variable("ECF_HOME","/user/var/home") suite.add_variable("ECF_INCLUDE","/user/var/home/includes") for i in range(1,7) : fam = suite.add_family("f" + str(i)) for t in ( "a", "b", "c", "d", "e" ) : fam.add_task(t); defs.generate_scripts() # generate ".ecf" and head.h/tail.h if required
get_all_nodes((Defs)arg1) → NodeVec :
Returns all the node s in the definition
get_all_tasks((Defs)arg1) → TaskVec :
Returns all the task nodes
get_server_state((Defs)arg1) → SState :
Returns the ecflow_server state: See server states
Usage:
try: ci = Client() # use default host(ECF_HOST) & port(ECF_PORT) ci.shutdown_server() ci.sync_local() assert ci.get_defs().get_server_state() == SState.SHUTDOWN, "Expected server to be shutdown" except RuntimeError, e: print(str(e))
get_state((Defs)arg1) → State
has_time_dependencies((Defs)arg1) → bool :
returns True if the suite definition has any time dependencies
restore_from_checkpt((Defs)arg1, (str)arg2) → None :
Restore the suite definition from a check point file stored on disk
save_as_checkpt((Defs)arg1, (str)arg2) → None :
Save the in memory suite definition as a check point file. This includes all node state.
save_as_defs((Defs)arg1, (str)arg2[, (Style)arg3]) → None :
Save the in memory suite definition into a file. The file name must be passed as an argument
simulate((Defs)arg1) → str :
Simulates a suite definition, allowing you predict/verify the behaviour of your suite in few seconds
The simulator will analyse the definition, and simulate the ecflow server. Allowing time dependencies that span several months, to be simulated in a few seconds. Ecflow allows the use of verify attributes. This example show how we can verify the number of times a task should run, given a start(optional) and end time(optional):
suite cron3 # use real clock otherwise clock starts when the simulations starts. clock real 1.1.2006 # define a start date for deterministic behaviour endclock 13.1.2006 # When to finish. end clock is *only* used for the simulator family cronFamily task t cron -d 10,11,12 10:00 11:00 01:00 # run on 10,11,12 of the month at 10am and 11am verify complete:6 # task should complete 6 times between 1.1.2006 -> 13.1.2006 endfamily endsuitePlease note, for deterministic behaviour, the start and end clock should be specified. However if no ‘endclock’ is specified the simulation will assume the following defaults.
- No time dependencies: 24 hours
- time || today : 24 hours
- day : 1 week
- date : 1 month
- cron : 1 year
- repeat : 1 year
If there no time dependencies with an minute resolution, then the simulator will by default use 1 hour resolution. This needs to be taken into account when specifying the verify attribute If the simulation does not complete it creates defs.flat and defs.depth files. This provides clues as to the state of the definition at the end of the simulation
Usage:
defs = Defs("my.def") # specify the defs we want to simulate .... theResults = defs.simulate() print(theResults)
sort_attributes((Defs)arg1, (str)attribute_type[, (bool)recursive=True]) → None
sort_attributes( (Defs)arg1, (AttrType)attribute_type [, (bool)recursive=True]) -> None
externs Returns a list of extern s server_variables Returns a list of server variable s suites Returns a list of suite s user_variables Returns a list of user defined variable s
class ecflow.Defstatus
Bases:
Boost.Python.instanceA node can be set with a default status other the queued
The default state of a node is queued. This defines the state to take at ‘begin’ or ‘re-queue’ time See
ecflow.Node.add_defstatusandecflow.DStatestate((Defstatus)arg1) → DState
class ecflow.Ecf
Bases:
Boost.Python.instanceSingleton used to control ecf debugging
static debug_equality() → bool :
Returns true if debugging of equality is enabled
static debug_level() → int :
Returns integer showing debug level. debug_level > 0 will disable some warning messages
static set_debug_equality((bool)arg1) → None :
Set debugging for equality
static set_debug_level((int)arg1) → None :
Set debug level. debug_level > 0 will disable some warning messages
class ecflow.Edit
Bases:
Boost.Python.instanceDefines a variable on a node for use in ecf script.
A Node can have a number of variables. These variables can be added at any node level: Defs, suite, family or task. The variables are names inside a pair of ‘%’ characters in an ecf script. The content of a variable replaces the variable name in the ecf script at job submission time. When a variable is needed at submission time, it is first sought in the task itself. If it is not found, it is sought from the tasks parent and so on, up through the node levels until found. See variable inheritance A undefined variable in a ecf script, causes the task to be aborted, without the job being submitted.
Constructor:
Variable(name,value) string name: the name of the variable string value: The value of the variable Edit(dict,kwarg) # alternative that allows multiple variables
Usage:
... var = Variable("ECF_JOB_CMD","/bin/sh %ECF_JOB% &") task.add_variable(var) task.add_variable("JOE","90")The following use example of using Edit, which allow multiple variables to added at the same time
t = Task("t1", Edit({ "a":"y", "b":"bb"}, c="v",d="b"), Edit({ "e":1100, "f":"bb"}), Edit(g="d"), Edit(h="1"))defs = Defs( Suite("s1"), Edit(SLEEP="1")) # Add user variable to definition defs.s1 += [ Task("a") ] defs.s1.a += [ Edit({ "x1":"y", "aa1":"bb"}, a="v",b="b"), Edit({ "var":10, "aa":"bb"}), Edit(d="d") ]
class ecflow.Event
Bases:
Boost.Python.instanceevent s are used as signal mechanism.
Typically they would be used to signal partial completion of a task and to be able to trigger another job, which is waiting for this partial completion. Only tasks can have events that are automatically set via a child command s, see below. Events are cleared automatically when a node is re-queued or begun. Suites and Families can have events, but these events must be set via the Alter command Multiple events can be added to a task. An Event has a number and a optional name. Events are typically used in trigger and complete expression , to control job creation. Event are fired within a script/job file, i.e...:
ecflow_client --init=$$ ecflow_client --event=foo ecflow_client --complete
Hence the defining of an event for a task, should be followed with the addition of ecflow_client –event child command in the corresponding ecf script file.
Constructor:
Event(number, optional<name = "">) int number : The number must be >= 0 string name<optional> : If name is given, can only refer to Event by its name
Usage:
event = Event(2,"event_name") task.add_event(event) task1.add_event("2") # create a event "1" and add to the task task2.add_event("name") # create a event "name" and add to task # Events can be created in the Task constructor, like any other attribute t = Task("t3", Event(2,"event_name"))empty((Event)arg1) → bool :
Return true if the Event is empty. Used when returning a NULL Event, from a find
name((Event)arg1) → str :
Return the Events name as string. If number supplied name may be empty.
name_or_number((Event)arg1) → str :
returns name or number as an string
number((Event)arg1) → int :
Return events number as a integer. If not specified return max integer value
value((Event)arg1) → bool :
Return events current value
class ecflow.Expression
Bases:
Boost.Python.instanceExpression holds trigger or complete expression. Also see
ecflow.TriggerExpressions can contain references to events, meters, user variables,repeat variables and generated variables. Expressions hold a list of part expressions. This allows us to split a large trigger or complete expression into smaller ones.
Constructor:
Expression( expression ) string expression : This typically represents the complete expression however part expression can still be added Expression( part ) PartExpression part: The first part expression should have no "and/or" setUsage: To add simple expression this class can be by passed, i.e... can use:
task = Task("t1") task.add_trigger( "t2 == active" ) task.add_complete( "t2 == complete" ) task = Task("t2") task.add_trigger( "t1 == active" ) task.add_part_trigger( "t3 == active", True)To store and add large expressions use a Expression with PartExpression:
big_expr = Expression( PartExpression("t1 == complete or t4 == complete") ) big_expr.add( PartExpression("t5 == active",True) ) big_expr.add( PartExpression("t7 == active",False) ) task.add_trigger( big_expr)In the example above the trigger for task is equivalent to ‘t1 == complete or t4 == complete and t5 == active or t7 == active’
big_expr2 = Expression("t0 == complete")) big_expr2.add( PartExpression("t1 == complete or t4 == complete",True) ) big_expr2.add( PartExpression("t5 == active",False) ) task2.add_trigger( big_expr2)Here the trigger for task2 is equivalent to ‘t0 == complete and t1 == complete or t4 == complete or t5 == active’
add((Expression)arg1, (PartExpression)arg2) → None :
Add a part expression, the second and subsequent part expressions must have ‘and/or’ set
get_expression((Expression)arg1) → str :
returns the complete expression as a string
parts Returns a list of PartExpression’s
class ecflow.Family
Bases:
ecflow.NodeContainerCreate a family node.A Family node lives inside a suite or another family
A family is used to collect task s together or to group other families. Typically you place tasks that are related to each other inside the same family analogous to the way you create directories to contain related files. There are two ways of adding a family, see example below.
Constructor:
Family(name, Nodes | Attributes) string name : The Family name. name must consist of alpha numeric characters or underscore or dot. The first character can not be dot, as this will interfere with trigger expressions. Case is significant Nodes | Attributes: (optional)Exception:
- Throws a RuntimeError if the name is not valid
- Throws a RuntimeError if a duplicate family is added
Usage:
suite = Suite("suite_1") # create a suite family = Family("family_1") # create a family suite.add_family(family) # add created family to a suite f2 = suite.add_family("f2") # create a family f2 and add to suite # create in place defs = Defs( Suite("s1", Family("f1", Task("t1", Edit(SLEEP="10")))))
class ecflow.FamilyVec
Bases:
Boost.Python.instanceHold a list of family nodes
append((FamilyVec)arg1, (object)arg2) → None
extend((FamilyVec)arg1, (object)arg2) → None
class ecflow.File
Bases:
Boost.Python.instanceUtility class, Used in test only.
static build_dir() → str :
Path name to ecflow build directory
static find_client() → str :
Provides pathname to the client
static find_server() → str :
Provides pathname to the server
static source_dir() → str :
Path name to ecflow source directory
class ecflow.Flag
Bases:
Boost.Python.instanceRepresents additional state associated with a Node.
clear((Flag)arg1, (FlagType)arg2) → None :
Clear the given flag. Used in test only
is_set((Flag)arg1, (FlagType)arg2) → bool :
Queries if a given flag is set
static list() → FlagTypeVec :
Returns the list of all flag types. returns FlagTypeVec. Used in test only
reset((Flag)arg1) → None :
Clears all flags. Used in test only
set((Flag)arg1, (FlagType)arg2) → None :
Sets the given flag. Used in test only
static type_to_string((FlagType)arg1) → str :
Convert type to a string. Used in test only
class ecflow.FlagType
Bases:
Boost.Python.enumFlags store state associated with a node
- FORCE_ABORT - Node* do not run when try_no > ECF_TRIES, and task aborted by user
- USER_EDIT - task
- TASK_ABORTED - task*
- EDIT_FAILED - task*
- JOBCMD_FAILED - task*
- NO_SCRIPT - task*
- KILLED - task* do not run when try_no > ECF_TRIES, and task killed by user
- MIGRATED - Node
- LATE - Node attribute, Task is late, or Defs checkpt takes to long
- MESSAGE - Node
- BYRULE - Node*, set if node is set to complete by complete trigger expression
- QUEUELIMIT - Node
- WAIT - task*
- LOCKED - Server
- ZOMBIE - task*
- NO_REQUE - task
- NOT_SET
byrule = ecflow.FlagType.byrule
edit_failed = ecflow.FlagType.edit_failed
force_abort = ecflow.FlagType.force_abort
jobcmd_failed = ecflow.FlagType.jobcmd_failed
killed = ecflow.FlagType.killed
late = ecflow.FlagType.late
locked = ecflow.FlagType.locked
message = ecflow.FlagType.message
migrated = ecflow.FlagType.migrated
names = {'migrated': ecflow.FlagType.migrated, 'queuelimit': ecflow.FlagType.queuelimit, 'user_edit': ecflow.FlagType.user_edit, 'zombie': ecflow.FlagType.zombie, 'edit_failed': ecflow.FlagType.edit_failed, 'task_aborted': ecflow.FlagType.task_aborted, 'late': ecflow.FlagType.late, 'force_abort': ecflow.FlagType.force_abort, 'not_set': ecflow.FlagType.not_set, 'no_reque': ecflow.FlagType.no_reque, 'no_script': ecflow.FlagType.no_script, 'locked': ecflow.FlagType.locked, 'message': ecflow.FlagType.message, 'byrule': ecflow.FlagType.byrule, 'jobcmd_failed': ecflow.FlagType.jobcmd_failed, 'killed': ecflow.FlagType.killed, 'wait': ecflow.FlagType.wait}
no_reque = ecflow.FlagType.no_reque
no_script = ecflow.FlagType.no_script
not_set = ecflow.FlagType.not_set
queuelimit = ecflow.FlagType.queuelimit
task_aborted = ecflow.FlagType.task_aborted
user_edit = ecflow.FlagType.user_edit
values = {0: ecflow.FlagType.force_abort, 1: ecflow.FlagType.user_edit, 2: ecflow.FlagType.task_aborted, 3: ecflow.FlagType.edit_failed, 4: ecflow.FlagType.jobcmd_failed, 5: ecflow.FlagType.no_script, 6: ecflow.FlagType.killed, 7: ecflow.FlagType.migrated, 8: ecflow.FlagType.late, 9: ecflow.FlagType.message, 10: ecflow.FlagType.byrule, 11: ecflow.FlagType.queuelimit, 12: ecflow.FlagType.wait, 13: ecflow.FlagType.locked, 14: ecflow.FlagType.zombie, 15: ecflow.FlagType.no_reque, 16: ecflow.FlagType.not_set}
wait = ecflow.FlagType.wait
zombie = ecflow.FlagType.zombie
class ecflow.FlagTypeVec
Bases:
Boost.Python.instanceHold a list of flag types
append((FlagTypeVec)arg1, (object)arg2) → None
extend((FlagTypeVec)arg1, (object)arg2) → None
class ecflow.InLimit
Bases:
Boost.Python.instanceinlimit is used in conjunction with limit to provide simple load management:
suite x limit fast 1 family f inlimit /x:fast task t1 task t2Here ‘fast’ is the name of
ecflow.Limitand the number defines the maximum number of tasks that can run simultaneously using this limit. Thats why you do not need a trigger between tasks ‘t1’ and ‘t2’. There is no need to change the tasks. The jobs are created in the order they are definedConstructor:
InLimit(name, optional<path = "">, optional<token = 1>) string name : The name of the referenced Limit string path<optional> : The path to the Limit, if this is left out, then Limit of "name" must be specified some where up the parent hierarchy int value<optional> : The usage of the Limit. Each job submission will consume "value" tokens from the Limit. defaults to 1 if no value specified.Usage:
inlimit = InLimit("fast","/x/f", 2) ... family = Family("f1", InLimit("mars","/x/f", 2)) # create InLimit in Node constructor family.add_inlimit(inlimit) # add existing inlimit using functionname((InLimit)arg1) → str :
Return the inlimit name as string
path_to_node((InLimit)arg1) → str :
Path to the node that holds the limit, can be empty
tokens((InLimit)arg1) → int :
The number of token to consume from the Limit
class ecflow.JobCreationCtrl
Bases:
Boost.Python.instanceThe class JobCreationCtrl is used in job creation checking
Constructor:
JobCreationCtrl()
Usage:
defs = Defs("my.def") # specify the definition we want to check, load into memory job_ctrl = JobCreationCtrl() job_ctrl.set_node_path("/suite/to_check") # will hierarchically check job creation under this node defs.check_job_creation(job_ctrl) # job files generated to ECF_JOB print(job_ctrl.get_error_msg()) # report any errors in job generation job_ctrl = JobCreationCtrl() # no set_node_path() hence check job creation for all tasks job_ctrl.set_dir_for_job_creation(tmp) # generate jobs file under this directory defs.check_job_creation(job_ctrl) print(job_ctrl.get_error_msg()) job_ctrl = JobCreationCtrl() # no set_node_path() hence check job creation for all tasks job_ctrl.generate_temp_dir() # automatically generate directory for job file defs.check_job_creation(job_ctrl) print(job_ctrl.get_error_msg())generate_temp_dir((JobCreationCtrl)arg1) → None :
Automatically generated temporary directory for job creation. Directory written to stdout for information
get_dir_for_job_creation((JobCreationCtrl)arg1) → str :
Returns the directory set for job creation
get_error_msg((JobCreationCtrl)arg1) → str :
Returns an error message generated during checking of job creation
set_dir_for_job_creation((JobCreationCtrl)arg1, (str)arg2) → None :
Specify directory, for job creation
set_node_path((JobCreationCtrl)arg1, (str)arg2) → None :
The node we want to check job creation for. If no node specified check all tasks
set_verbose((JobCreationCtrl)arg1, (bool)arg2) → None :
Output each task as its being checked.
class ecflow.Label
Bases:
Boost.Python.instanceA label has a name and value and provides a way of displaying information in a GUI.
The value can be anything(ASCII) as it can not be used in triggers. The value of the label is set to be the default value given in the definition when the suite is begun. This is useful in repeated suites: A task sets the label to be something.
Labels can be set at any level: Suite,Family,Task. There are two ways of updating the label
- A child command can be used to automatically update the label on a task
- Using the alter command, the labels on suite family and task can be changed manually
Constructor:
Label(name,value) string name: The name of the label string value: The value of the label
Usage:
t1 = Task("t1", Label("name","value"), # create Labels in-place Label("a","b")) t1.add_label("l1","value") t1.add_label(Label("l2","value2")) for label in t1.labels: print(label)empty((Label)arg1) → bool :
Return true if the Label is empty. Used when returning a NULL Label, from a find
name((Label)arg1) → str :
Return the label name as string
new_value((Label)arg1) → str :
Return the new label value as string
value((Label)arg1) → str :
Return the original label value as string
class ecflow.Late
Bases:
Boost.Python.instanceSets the late flag.
When a Node is classified as being late, the only action ecflow_server can take is to set a flag. The GUI will display this alongside the node name as a icon. Only one Late attribute can be specified on a Node.
Constructor:
Late() Late(kwargs)
Usage:
# This is interpreted as: The node can stay `submitted`_ for a maximum of 15 minutes # and it must become `active`_ by 20:00 and the run time must not exceed 2 hours late = Late() late.submitted( 0,15 ) late.active( 20,0 ) late.complete( 2,0, true ) late = Late(submitted="00:15",active="20:00",complete="+02:00") t = Task("t1", Late(submitted="00:15",active="20:00"))active((Late)arg1, (int)arg2, (int)arg3) → None :
- active(hour,minute): The time the node must become active. If the node is still queued or submitted
- by the time specified, the late flag is set
- active( (Late)arg1, (TimeSlot)arg2) -> None :
- active(TimeSlot):The time the node must become active. If the node is still queued or submitted by the time specified, the late flag is set
- active( (Late)arg1) -> TimeSlot :
- Return the active time as a TimeSlot
complete((Late)arg1, (int)arg2, (int)arg3, (bool)arg4) → None :
- complete(hour,minute):The time the node must become complete. If relative, time is taken from the time
- the node became active, otherwise node must be complete by the time given
- complete( (Late)arg1, (TimeSlot)arg2, (bool)arg3) -> None :
- complete(TimeSlot): The time the node must become complete. If relative, time is taken from the time the node became active, otherwise node must be complete by the time given
- complete( (Late)arg1) -> TimeSlot :
- Return the complete time as a TimeSlot
complete_is_relative((Late)arg1) → bool :
Returns a boolean where true means that complete is relative
is_late((Late)arg1) → bool :
Return True if late
submitted((Late)arg1, (TimeSlot)arg2) → None :
- submitted(TimeSlot):The time node can stay submitted. Submitted is always relative. If the node stays
- submitted longer than the time specified, the late flag is set
- submitted( (Late)arg1, (int)arg2, (int)arg3) -> None :
- submitted(hour,minute) The time node can stay submitted. Submitted is always relative. If the node stays submitted longer than the time specified, the late flag is set
- submitted( (Late)arg1) -> TimeSlot :
- Return the submitted time as a TimeSlot
class ecflow.Limit
Bases:
Boost.Python.instancelimit provides a simple load management
i.e... by limiting the number of task s submitted by a server. Limits are typically defined at the suite level, or defined in a separate suite, so that they can be used by multiple suites. Once a limit is defined in a suite definition, you must also assign families/tasks to use this limit. See inlimit and
ecflow.InLimitConstructor:
Limit(name,value) string name: the name of the limit int value: The value of the limit
Usage:
limit = Limit("fast", 10) ... suite = Suite("s1", Limit("slow",10)) # create Limit in Node constructor suite.add_limit(limit) # add existing limit using functiondecrement((Limit)arg1, (int)arg2, (str)arg3) → None :
used for test only
increment((Limit)arg1, (int)arg2, (str)arg3) → None :
used for test only
limit((Limit)arg1) → int :
The max value of the limit as an integer
name((Limit)arg1) → str :
Return the limit name as string
node_paths((Limit)arg1) → list :
List of nodes(paths) that have consumed a limit
value((Limit)arg1) → int :
The limit token value as an integer
class ecflow.Meter
Bases:
Boost.Python.instancemeter s can be used to indicate proportional completion of task
They are able to trigger another job, which is waiting on this proportion. Can also be used to indicate progress of a job. Meters can be used in trigger and complete expression.
Constructor:
Meter(name,min,max,<optional>color_change) string name : The meter name int min : The minimum and initial meter value int max : The maximum meter value. Must be greater than min value. int color_change<optional> : default = max, Must be between min-max, used in the GUI
Exceptions:
- raises IndexError when an invalid Meter is specified
Usage:
Using a meter requires:
Defining a meter on a task:
meter = Meter("progress",0,100,100) task.add_meter(meter)Updating the corresponding ecf script file with the meter child command:
ecflow_client --init=$$ for i in 10 20 30 40 50 60 80 100; do ecflow_client --meter=progress $i sleep 2 # or do some work done ecflow_client --completeOptionally addition in a trigger or complete expression for job control:
trigger task:progress ge 60
trigger and complete expression should avoid using equality i.e..:
trigger task:progress == 60
Due to network issues the meter event’s may not arrive in sequential order hence the ecflow_server will ignore meter value’s, which are less than the current value as a result triggers’s which use meter equality may never evaluate
color_change((Meter)arg1) → int :
returns the color change
empty((Meter)arg1) → bool :
Return true if the Meter is empty. Used when returning a NULL Meter, from a find
max((Meter)arg1) → int :
Return the Meters maximum value
min((Meter)arg1) → int :
Return the Meters minimum value
name((Meter)arg1) → str :
Return the Meters name as string
value((Meter)arg1) → int :
Return meters current value
class ecflow.Node
Bases:
Boost.Python.instanceA Node class is the abstract base class for Suite, Family and Task
Every Node instance has a name, and a path relative to a suite
add()
- object add(tuple args, dict kwds) :
add(..) provides a way to append Nodes and attributes
This is best illustrated with an example:
defs = Defs().add( Suite("s1").add( Clock(1, 1, 2010, False), Autocancel(1, 10, True), Task("t1").add( Edit({"a":"12", "b":"bb"}, c="v",d="b"), Edit(g="d"), Edit(h=1), Event(1), Event(11,"event"), Meter("meter",0,10,10), Label("label","c"), Trigger("1==1"), Complete("1==1"), Limit("limit",10),Limit("limit2",10), InLimit("limitName","/limit",2), Defstatus(DState.complete), Today(0,30),Today("00:59"),Today("00:00 11:30 00:01"), Time(0,30),Time("00:59"),Time("00:00 11:30 00:01"), Day("sunday"),Day(Days.monday), Date(1,1,0),Date(28,2,1960), Autocancel(3) ), [ Family("f{}".format(i)) for i in range(1,6)]))We can also use ‘+=’ with a list here are a few examples:
defs = Defs(); defs += [ Suite("s2"),Edit({ "x1":"y", "aa1":"bb"}, a="v",b="b") ]defs += [ Suite("s{}".format(i)) for i in range(1,6) ]defs = Defs() defs += [ Suite("suite").add( Task("x"), Family("f").add( [ Task("t{}".format(i)) for i in range(1,6)] ), Task("y"), [ Family("f{}".format(i)) for i in range(1,6) ], Edit(a="b"), [ Task("t{}".format(i)) for i in range(1,6) ], )]It is also possible to use ‘+’
defs = Defs() + Suite("s1") defs.s1 += Autocancel(1, 10, True) defs.s1 += Task("t1") + Edit({ "e":1, "f":"bb"}) +\ Event(1) + Event(11,"event") + Meter("meter",0,10,10) + Label("label","c") + Trigger("1==1") +\ Complete("1==1") + Limit("limit",10) + Limit("limit2",10) + InLimit("limitName","/limit",2) +\ Defstatus(DState.complete) + Today(0,30) + Today("00:59") + Today("00:00 11:30 00:01") +\ Time(0,30) + Time("00:59") + Time("00:00 11:30 00:01") + Day("sunday") + Day(Days.monday) +\ Date(1,1,0) + Date(28,2,1960) + Autocancel(3)Warning
We can only use ‘+’ when the left most object is a node, i.e.. Task(‘t1’) in this case
add_autocancel((Node)arg1, (int)arg2) → Node :
Add a autocancel attribute. See
ecflow.AutocancelThis will delete the node on completion. The deletion may be delayed by an amount of time in hours and minutes or expressed as days Node deletion is not immediate. The nodes are checked once a minute and expired auto cancel nodes are deleted A node may only have one auto cancel attribute
Exception:
- Throws a RuntimeError if more than one auto cancel is added
Usage:
t1 = Task("t1") t1.add_autocancel( Autocancel(20,10,False) ) # hour,min, relative t2 = Task("t2") t2.add_autocancel( 3 ) # 3 days t3 = Task("t3") t3.add_autocancel( 20,10,True ) # hour,minutes,relative t4 = Task("t4") t4.add_autocancel( TimeSlot(20,10),True ) # hour,minutes,relative # we can also create a Autocancel in the Task constructor like any other attribute t2 = Task("t2", Autocancel(20,10,False))add_autocancel( (Node)arg1, (int)arg2, (int)arg3, (bool)arg4) -> Node
add_autocancel( (Node)arg1, (TimeSlot)arg2, (bool)arg3) -> Node
add_autocancel( (Node)arg1, (Autocancel)arg2) -> Node
add_complete((Node)arg1, (str)arg2) → Node :
Add a trigger or complete expression.Also see
ecflow.TriggerThis defines a dependency for a node. There can only be one trigger or complete expression dependency per node. A node with a trigger can only be activated when the trigger has expired. A trigger holds a node as long as the expression returns false.
Exception:
- Will throw RuntimeError if multiple trigger or complete expression are added
- Will throw RuntimeError if first expression is added as ‘AND’ or ‘OR’ expression Like wise second and subsequent expression must have ‘AND’ or ‘OR’ booleans set
Usage:
Note we can not make multiple add_trigger(..) calls on the same task! to add a simple trigger:
task1.add_trigger( "t2 == active" ) task2.add_trigger( "t1 == complete or t4 == complete" ) task3.add_trigger( "t5 == active" )
Long expression can be broken up using add_part_trigger:
task2.add_part_trigger( "t1 == complete or t4 == complete") task2.add_part_trigger( "t5 == active",True) # True means AND task2.add_part_trigger( "t7 == active",False) # False means OR
The trigger for task2 is equivalent to: ‘t1 == complete or t4 == complete and t5 == active or t7 == active’
add_complete( (Node)arg1, (Expression)arg2) -> Node
add_cron((Node)arg1, (Cron)arg2) → Node :
Add a cron time dependency. See
ecflow.CronUsage:
start = TimeSlot(0,0) finish = TimeSlot(23,0) incr = TimeSlot(0,30) time_series = TimeSeries( start, finish, incr, True) cron = Cron() cron.set_week_days( [0,1,2,3,4,5,6] ) cron.set_days_of_month( [1,2,3,4,5,6] ) cron.set_months( [1,2,3,4,5,6] ) cron.set_time_series( time_series ) t1 = Task("t1") t1.add_cron( cron ) # we can also create a Cron in the Task constructor like any other attribute t2 = Task("t2", Cron("+00:00 23:00 00:30",days_of_week=[0,1,2,3,4,5,6],days_of_month=[1,2,3,4,5,6],months=[1,2,3,4,5,6]))
add_date((Node)arg1, (int)arg2, (int)arg3, (int)arg4) → Node :
Add a date time dependency. See
ecflow.DateA value of zero for day,month,year means every day, every month, every year
Exception:
- Throws RuntimeError if an invalid date is added
Usage:
t1 = Task("t1", Date("1.*.*"), Date(1,1,2010))) # Create Date in place t1.add_date( Date(1,1,2010) ) # day,month,year t1.add_date( 2,1,2010) # day,month,year t1.add_date( 1,0,0) # day,month,year, the first of each month for every yearadd_date( (Node)arg1, (Date)arg2) -> Node
add_day((Node)arg1, (Days)arg2) → Node :
Add a day time dependency. See
ecflow.DayUsage:
t1 = Task("t1", Day("sunday")) # Create Day on Task creation t1.add_day( Day(Days.sunday) ) t1.add_day( Days.monday) t1.add_day( "tuesday" )add_day( (Node)arg1, (str)arg2) -> Node
add_day( (Node)arg1, (Day)arg2) -> Node
add_defstatus((Node)arg1, (DState)arg2) → Node :
Set the default status( defstatus ) of node at begin or re queue. See
ecflow.DefstatusA defstatus is useful in preventing suites from running automatically once begun, or in setting Task’s complete so they can be run selectively
Usage:
t1 = Task("t1") + Defstatus("complete") t2 = Task("t2").add_defstatus( DState.suspended ) # we can also create a Defstatus in the Task constructor like any other attribute t2 = Task("t3", Defstatus("complete"))- add_defstatus( (Node)arg1, (Defstatus)arg2) -> Node :
Set the default status( defstatus ) of node at begin or re queue. See
ecflow.DefstatusA defstatus is useful in preventing suites from running automatically once begun, or in setting Task’s complete so they can be run selectively
Usage:
t1 = Task("t1") + Defstatus("complete") t2 = Task("t2").add_defstatus( DState.suspended ) # we can also create a Defstatus in the Task constructor like any other attribute t2 = Task("t3", Defstatus("complete"))
add_event((Node)arg1, (Event)arg2) → Node :
- Add a event. See
ecflow.Event Events can be referenced in trigger and complete expression s
Exception:
- Throws RuntimeError if a duplicate is added
Usage:
t1 = Task("t1", Event(12), Event(11,"eventx")) # Create events on Task creation t1.add_event( Event(10) ) # Create with function on Task t1.add_event( Event(11,"Eventname") ) t1.add_event( 12 ) t1.add_event( 13, "name")To reference event ‘flag’ in a trigger:
t1.add_event("flag") t2 = Task("t2", Trigger("t1:flag == set"))
add_event( (Node)arg1, (int)arg2) -> Node
add_event( (Node)arg1, (int)arg2, (str)arg3) -> Node
add_event( (Node)arg1, (str)arg2) -> Node
- Add a event. See
add_inlimit((Node)arg1, (str)limit_name[, (str)path_to_node_containing_limit=''[, (int)tokens=1]]) → Node :
Adds a inlimit to a node. See
ecflow.InLimitInLimit reference a limit/
ecflow.Limit. Duplicate InLimits are not allowedException:
- Throws RuntimeError if a duplicate is added
Usage:
task2.add_inlimit( InLimit("limitName","/s1/f1",2) ) task2.add_inlimit( "limitName","/s1/f1",2 )add_inlimit( (Node)arg1, (InLimit)arg2) -> Node
add_label((Node)arg1, (str)arg2, (str)arg3) → Node :
Adds a label to a node. See
ecflow.LabelLabels can be updated from the jobs files, via child command
Exception:
- Throws RuntimeError if a duplicate label name is added
Usage:
task.add_label( Label("TEA","/me/")) task.add_label( "Joe","/me/")The corresponding child command in the .ecf script file might be:
ecflow_client --label=TEA time ecflow_client --label=Joe ninety
add_label( (Node)arg1, (Label)arg2) -> Node
add_late((Node)arg1, (Late)arg2) → Node :
Add a late attribute. See
ecflow.LateException:
- Throws a RuntimeError if more than one late is added
Usage:
late = Late() late.submitted( 20,10 ) # hour,minute late.active( 20,10 ) # hour,minute late.complete( 20,10,True) # hour,minute,relative t1 = Task("t1") t1.add_late( late ) # we can also create a Late in the Task constructor like any other attribute t2 = Task("t2", Late(submitted="20:10",active="20:10",complete="+20:10"))
add_limit((Node)arg1, (str)arg2, (int)arg3) → Node :
Adds a limit to a node for simple load management. See
ecflow.LimitMultiple limits can be added, however the limit name must be unique. For a node to be in a limit, a inlimit must be used.
Exception:
- Throws RuntimeError if a duplicate limit name is added
Usage:
family.add_limit( Limit("load",12) ) family.add_limit( "load",12 )add_limit( (Node)arg1, (Limit)arg2) -> Node
add_meter((Node)arg1, (Meter)arg2) → Node :
- Add a meter. See
ecflow.Meter Meters can be referenced in trigger and complete expression s
Exception:
- Throws RuntimeError if a duplicate is added
Usage:
t1 = Task("t1", Meter("met",0,50)) # create Meter on Task creation t1.add_meter( Meter("metername",0,100,50) ) # create Meter using function t1.add_meter( "meter",0,200)To reference in a trigger:
t2 = Task("t2") t2.add_trigger("t1:meter >= 10")
add_meter( (Node)arg1, (str)arg2, (int)arg3, (int)arg4 [, (int)arg5]) -> Node
- Add a meter. See
add_part_complete((Node)arg1, (PartExpression)arg2) → Node :
Add a trigger or complete expression.Also see
ecflow.TriggerThis defines a dependency for a node. There can only be one trigger or complete expression dependency per node. A node with a trigger can only be activated when the trigger has expired. A trigger holds a node as long as the expression returns false.
Exception:
- Will throw RuntimeError if multiple trigger or complete expression are added
- Will throw RuntimeError if first expression is added as ‘AND’ or ‘OR’ expression Like wise second and subsequent expression must have ‘AND’ or ‘OR’ booleans set
Usage:
Note we can not make multiple add_trigger(..) calls on the same task! to add a simple trigger:
task1.add_trigger( "t2 == active" ) task2.add_trigger( "t1 == complete or t4 == complete" ) task3.add_trigger( "t5 == active" )
Long expression can be broken up using add_part_trigger:
task2.add_part_trigger( "t1 == complete or t4 == complete") task2.add_part_trigger( "t5 == active",True) # True means AND task2.add_part_trigger( "t7 == active",False) # False means OR
The trigger for task2 is equivalent to: ‘t1 == complete or t4 == complete and t5 == active or t7 == active’
add_part_complete( (Node)arg1, (str)arg2) -> Node
add_part_complete( (Node)arg1, (str)arg2, (bool)arg3) -> Node
add_part_trigger((Node)arg1, (PartExpression)arg2) → Node :
Add a trigger or complete expression.Also see
ecflow.TriggerThis defines a dependency for a node. There can only be one trigger or complete expression dependency per node. A node with a trigger can only be activated when the trigger has expired. A trigger holds a node as long as the expression returns false.
Exception:
- Will throw RuntimeError if multiple trigger or complete expression are added
- Will throw RuntimeError if first expression is added as ‘AND’ or ‘OR’ expression Like wise second and subsequent expression must have ‘AND’ or ‘OR’ booleans set
Usage:
Note we can not make multiple add_trigger(..) calls on the same task! to add a simple trigger:
task1.add_trigger( "t2 == active" ) task2.add_trigger( "t1 == complete or t4 == complete" ) task3.add_trigger( "t5 == active" )
Long expression can be broken up using add_part_trigger:
task2.add_part_trigger( "t1 == complete or t4 == complete") task2.add_part_trigger( "t5 == active",True) # True means AND task2.add_part_trigger( "t7 == active",False) # False means OR
The trigger for task2 is equivalent to: ‘t1 == complete or t4 == complete and t5 == active or t7 == active’
add_part_trigger( (Node)arg1, (str)arg2) -> Node
add_part_trigger( (Node)arg1, (str)arg2, (bool)arg3) -> Node
add_repeat((Node)arg1, (RepeatDate)arg2) → Node :
Add a RepeatDate attribute. See
ecflow.RepeatDateA node can only have one repeat
Exception:
- Throws a RuntimeError if more than one repeat is added
Usage:
t1 = Task("t1") t1.add_repeat( RepeatDate("testDate",20100111,20100115) ) # we can also create a repeat in Task constructor like any other attribute t2 = Task("t2", RepeatDate("testDate",20100111,20100115))- add_repeat( (Node)arg1, (RepeatInteger)arg2) -> Node :
Add a RepeatInteger attribute. See
ecflow.RepeatIntegerA node can only have one repeat
Exception:
- Throws a RuntimeError if more than one repeat is added
Usage:
t1 = Task("t1") t1.add_repeat( RepeatInteger("testInteger",0,100,2) ) # we can also create a repeat in Task constructor like any other attribute t2 = Task("t2", RepeatInteger("testInteger",0,100,2))- add_repeat( (Node)arg1, (RepeatString)arg2) -> Node :
Add a RepeatString attribute. See
ecflow.RepeatStringA node can only have one repeat
Exception:
- Throws a RuntimeError if more than one repeat is added
Usage:
t1 = Task("t1") t1.add_repeat( RepeatString("test_string",["a", "b", "c" ] ) ) # we can also create a repeat in Task constructor like any other attribute t2 = Task("t2", RepeatString("test_string",["a", "b", "c" ] ) )- add_repeat( (Node)arg1, (RepeatEnumerated)arg2) -> Node :
Add a RepeatEnumerated attribute. See
ecflow.RepeatEnumeratedA node can only have one repeat
Exception:
- Throws a RuntimeError if more than one repeat is added
Usage:
t1 = Task("t1") t1.add_repeat( RepeatEnumerated("test_string", ["red", "green", "blue" ] ) ) # we can also create a repeat in Task constructor like any other attribute t2 = Task("t2", RepeatEnumerated("test_string", ["red", "green", "blue" ] ) )- add_repeat( (Node)arg1, (RepeatDay)arg2) -> Node :
Add a RepeatDay attribute. See
ecflow.RepeatDayA node can only have one repeat
Exception:
- Throws a RuntimeError if more than one repeat is added
Usage:
t2 = Task("t2", RepeatDay(1))
add_time((Node)arg1, (int)arg2, (int)arg3) → Node :
Add a time dependency. See
ecflow.TimeUsage:
t1 = Task("t1", Time("+00:30 20:00 01:00")) # Create Time in Task constructor t1.add_time( "00:30" ) t1.add_time( "+00:30" ) t1.add_time( "+00:30 20:00 01:00" ) t1.add_time( Time( 0,10 )) # hour,min,relative =false t1.add_time( Time( 0,12,True )) # hour,min,relative t1.add_time( Time(TimeSlot(20,20),False)) t1.add_time( 0,1 )) # hour,min,relative=false t1.add_time( 0,3,False )) # hour,min,relative=false start = TimeSlot(0,0) finish = TimeSlot(23,0) incr = TimeSlot(0,30) ts = TimeSeries( start, finish, incr, True) task2.add_time( Time(ts) )add_time( (Node)arg1, (int)arg2, (int)arg3, (bool)arg4) -> Node
add_time( (Node)arg1, (str)arg2) -> Node
add_time( (Node)arg1, (Time)arg2) -> Node
add_today((Node)arg1, (int)arg2, (int)arg3) → Node :
Add a today time dependency. See
ecflow.TodayUsage:
t1 = Task("t1", Today("+00:30 20:00 01:00")) # Create Today in Task constructor t1.add_today( "00:30" ) t1.add_today( "+00:30" ) t1.add_today( "+00:30 20:00 01:00" ) t1.add_today( Today( 0,10 )) # hour,min,relative =false t1.add_today( Today( 0,12,True )) # hour,min,relative t1.add_today( Today(TimeSlot(20,20),False)) t1.add_today( 0,1 )) # hour,min,relative=false t1.add_today( 0,3,False )) # hour,min,relative=false start = TimeSlot(0,0) finish = TimeSlot(23,0) incr = TimeSlot(0,30) ts = TimeSeries( start, finish, incr, True) task2.add_today( Today(ts) )add_today( (Node)arg1, (int)arg2, (int)arg3, (bool)arg4) -> Node
add_today( (Node)arg1, (str)arg2) -> Node
add_today( (Node)arg1, (Today)arg2) -> Node
add_trigger((Node)arg1, (str)arg2) → Node :
Add a trigger or complete expression.Also see
ecflow.TriggerThis defines a dependency for a node. There can only be one trigger or complete expression dependency per node. A node with a trigger can only be activated when the trigger has expired. A trigger holds a node as long as the expression returns false.
Exception:
- Will throw RuntimeError if multiple trigger or complete expression are added
- Will throw RuntimeError if first expression is added as ‘AND’ or ‘OR’ expression Like wise second and subsequent expression must have ‘AND’ or ‘OR’ booleans set
Usage:
Note we can not make multiple add_trigger(..) calls on the same task! to add a simple trigger:
task1.add_trigger( "t2 == active" ) task2.add_trigger( "t1 == complete or t4 == complete" ) task3.add_trigger( "t5 == active" )
Long expression can be broken up using add_part_trigger:
task2.add_part_trigger( "t1 == complete or t4 == complete") task2.add_part_trigger( "t5 == active",True) # True means AND task2.add_part_trigger( "t7 == active",False) # False means OR
The trigger for task2 is equivalent to: ‘t1 == complete or t4 == complete and t5 == active or t7 == active’
add_trigger( (Node)arg1, (Expression)arg2) -> Node
add_variable((Node)arg1, (str)arg2, (str)arg3) → Node :
Adds a name value variable. Also see
ecflow.EditThis defines a variable for use in variable substitution in a ecf script file. There can be any number of variables. The variables are names inside a pair of ‘%’ characters in an ecf script. The name are case sensitive. Special character in the value, must be placed inside single quotes if misinterpretation is to be avoided. The value of the variable replaces the variable name in the ecf script at job creation time. The variable names for any given node must be unique. If duplicates are added then the the last value added is kept.
Exception:
- Writes warning to standard output, if a duplicate variable name is added
Usage:
task.add_variable( Variable("ECF_HOME","/tmp/")) task.add_variable( "TMPDIR","/tmp/") task.add_variable( "COUNT",2) a_dict = { "name":"value", "name2":"value2", "name3":"value3" } task.add_variable(a_dict)add_variable( (Node)arg1, (str)arg2, (int)arg3) -> Node
add_variable( (Node)arg1, (Variable)arg2) -> Node
add_variable( (Node)arg1, (dict)arg2) -> Node
add_verify((Node)arg1, (Verify)arg2) → None :
Add a Verify attribute.
- Used in python simulation used to assert that a particular state was reached. t2 = Task(‘t2’,
- Verify(State.complete, 6)) # verify task completes 6 times during simulation
add_zombie((Node)arg1, (ZombieAttr)arg2) → Node :
The zombie attribute defines how a zombie should be handled in an automated fashion
Very careful consideration should be taken before this attribute is added as it may hide a genuine problem. It can be added to any node. But is best defined at the suite or family level. If there is no zombie attribute the default behaviour is to block the init,complete,abort child command. and fob the event,label,and meter child command This attribute allows the server to make a automated response. Please see:
ecflow.ZombieType,ecflow.ChildCmdType,ecflow.ZombieUserActionTypeConstructor:
ZombieAttr(ZombieType,ChildCmdTypes, ZombieUserActionType, lifetime) ZombieType : Must be one of ZombieType.ecf, ZombieType.path, ZombieType.user ChildCmdType : A list(ChildCmdType) of Child commands. Can be left empty in which case the action affect all child commands ZombieUserActionType : One of [ fob, fail, block, remove, adopt ] int lifetime<optional>: Defines the life time in seconds of the zombie in the server. On expiration, zombie is removed automaticallyUsage:
# Add a zombie attribute so that child commands(i.e.. ecflow_client --init) # will fail the job if it is a zombie process. s1 = Suite("s1") child_list = [ ChildCmdType.init, ChildCmdType.complete, ChildCmdType.abort ] s1.add_zombie( ZombieAttr(ZombieType.ecf, child_list, ZombieUserActionType.fob)) # create the zombie as part of the node constructor s1 = Suite("s1", ZombieAttr(ZombieType.ecf, child_list, ZombieUserActionType.fail))
change_complete((Node)arg1, (str)arg2) → None
change_trigger((Node)arg1, (str)arg2) → None
delete_complete((Node)arg1) → None
delete_cron((Node)arg1, (str)arg2) → None
delete_cron( (Node)arg1, (Cron)arg2) -> None
delete_date((Node)arg1, (str)arg2) → None
delete_date( (Node)arg1, (Date)arg2) -> None
delete_day((Node)arg1, (str)arg2) → None
delete_day( (Node)arg1, (Day)arg2) -> None
delete_event((Node)arg1, (str)arg2) → None
delete_inlimit((Node)arg1, (str)arg2) → None
delete_label((Node)arg1, (str)arg2) → None
delete_limit((Node)arg1, (str)arg2) → None
delete_meter((Node)arg1, (str)arg2) → None
delete_repeat((Node)arg1) → None
delete_time((Node)arg1, (str)arg2) → None
delete_time( (Node)arg1, (Time)arg2) -> None
delete_today((Node)arg1, (str)arg2) → None
delete_today( (Node)arg1, (Today)arg2) -> None
delete_trigger((Node)arg1) → None
delete_variable((Node)arg1, (str)arg2) → None
delete_zombie((Node)arg1, (str)arg2) → None
delete_zombie( (Node)arg1, (ZombieType)arg2) -> None
evaluate_complete((Node)arg1) → bool :
evaluate complete expression
evaluate_trigger((Node)arg1) → bool :
evaluate trigger expression
find_event((Node)arg1, (str)arg2) → Event :
Find the event on the node only. Returns a object
find_gen_variable((Node)arg1, (str)arg2) → Variable :
Find generated variable on the node only. Returns a object
find_label((Node)arg1, (str)arg2) → Label :
Find the label on the node only. Returns a object
find_limit((Node)arg1, (str)arg2) → Limit :
Find the limit on the node only. returns a limit ptr
find_meter((Node)arg1, (str)arg2) → Meter :
Find the meter on the node only. Returns a object
find_node_up_the_tree((Node)arg1, (str)arg2) → Node :
Search immediate node, then up the node hierarchy
find_parent_variable((Node)arg1, (str)arg2) → Variable :
Find user variable variable up the parent hierarchy. Returns a object
find_variable((Node)arg1, (str)arg2) → Variable :
Find user variable on the node only. Returns a object
get_abs_node_path((Node)arg1) → str :
returns a string which holds the path to the node
get_all_nodes((Node)arg1) → NodeVec :
Returns all the child nodes
get_autocancel((Node)arg1) → Autocancel
get_complete((Node)arg1) → Expression
get_defs((Node)arg1) → Defs
get_defstatus((Node)arg1) → DState
get_dstate((Node)arg1) → DState :
Returns the state of node. This will include suspended state
get_flag((Node)arg1) → Flag :
Return additional state associated with a node.
get_generated_variables((Node)arg1, (VariableList)arg2) → None :
returns a list of generated variables. Use ecflow.VariableList as return argument
get_late((Node)arg1) → Late
get_parent((Node)arg1) → Node
get_repeat((Node)arg1) → Repeat
get_state((Node)arg1) → State :
Returns the state of the node. This excludes the suspended state
get_state_change_time((Node)arg1[, (str)format='iso_extended']) → str :
Returns the time of the last state change as a string. Default format is iso_extended, (iso_extended, iso, simple)
get_trigger((Node)arg1) → Expression
has_time_dependencies((Node)arg1) → bool
is_suspended((Node)arg1) → bool :
name((Node)arg1) → str
remove((Node)arg1) → Node :
Remove the node from its parent. and returns it
replace_on_server((Node)arg1[, (bool)suspend_node_first=True[, (bool)force=True]]) → None :
replace node on the server.
- replace_on_server( (Node)arg1, (str)arg2, (str)arg3 [, (bool)suspend_node_first=True [, (bool)force=True]]) -> None :
- replace node on the server.
- replace_on_server( (Node)arg1, (str)arg2 [, (bool)suspend_node_first=True [, (bool)force=True]]) -> None :
- replace node on the server.
sort_attributes((Node)arg1, (AttrType)attribute_type[, (bool)recursive=True]) → None
sort_attributes( (Node)arg1, (str)attribute_type [, (bool)recursive=True]) -> None
update_generated_variables((Node)arg1) → None
crons Returns a list of cron s dates Returns a list of date s days Returns a list of day s events Returns a list of event s inlimits Returns a list of inlimit s labels Returns a list of label s limits Returns a list of limit s meters Returns a list of meter s times Returns a list of time s todays Returns a list of today s variables Returns a list of user defined variable s verifies Returns a list of Verify’s zombies Returns a list of zombie s
class ecflow.NodeContainer
Bases:
ecflow.NodeNodeContainer is the abstract base class for a Suite and Family
A NodeContainer can have Families and Tasks as children
add_family((NodeContainer)arg1, (str)arg2) → Family :
Add a family. See
ecflow.Family.Multiple families can be added. However family names must be unique. for a given parent. Families can be hierarchical.
Exception:
- Throws RuntimeError if a duplicate is added
Usage:
suite = Suite("suite") # create a suite f1 = Family("f1") # create a family suite.add_family(f1) # add family to suite f2 = suite.add_family("f2") # create a family and add to suiteadd_family( (NodeContainer)arg1, (Family)arg2) -> Family
add_task((NodeContainer)arg1, (str)arg2) → Task :
Add a task. See
ecflow.TaskMultiple Tasks can be added. However Task names must be unique, for a given parent. Task can be added to Familiy’s or Suites.
Exception:
- Throws RuntimeError if a duplicate is added
Usage:
f1 = Family("f1") # create a family t1 = Task("t1") # create a task f1.add_task(t1) # add task to family t2 = f1.add_task("t2") # create task "t2" and add to familyadd_task( (NodeContainer)arg1, (Task)arg2) -> Task
find_family((NodeContainer)arg1, (str)arg2) → Family :
Find a family given a name
find_task((NodeContainer)arg1, (str)arg2) → Task :
Find a task given a name
nodes Returns a list of Node’s
class ecflow.NodeVec
Bases:
Boost.Python.instanceHold a list of Nodes (i.e.. suite, family or task s)
append((NodeVec)arg1, (object)arg2) → None
extend((NodeVec)arg1, (object)arg2) → None
class ecflow.PartExpression
Bases:
Boost.Python.instancePartExpression holds part of a trigger or complete expression.
Expressions can contain references to event, meter s, user variables, repeat variables and generated variables. The part expression allows us to split a large trigger or complete expression into smaller ones
Constructor:
PartExpression(exp ) string exp: This represents the *first* expression PartExpression(exp, bool and_expr) string exp: This represents the expression bool and_exp: If true the expression is to be anded, with a previously added expression If false the expression is to be "ored", with a previously added expressionUsage: To add simple expression this class can be by-passed, i.e... can use:
task = Task("t1") task.add_trigger( "t2 == active" ) task.add_complete( "t2 == complete" )To add large triggers and complete expression:
exp1 = PartExpression("t1 == complete") # a simple expression can be added as a string .... task2.add_part_trigger( PartExpression("t1 == complete or t4 == complete") ) task2.add_part_trigger( PartExpression("t5 == active",True) ) # anded with first expression task2.add_part_trigger( PartExpression("t7 == active",False) ) # or"ed with last expression addedThe trigger for task2 is equivalent to ‘t1 == complete or t4 == complete and t5 == active or t7 == active’
and_expr((PartExpression)arg1) → bool
get_expression((PartExpression)arg1) → str :
returns the part expression as a string
or_expr((PartExpression)arg1) → bool
class ecflow.PrintStyle
Bases:
Boost.Python.instanceSingleton used to control the print Style. See
ecflow.StyleUsage:
old_style = PrintStyle.get_style() PrintStyle.set_style(PrintStyle.STATE) ... print(defs) # show the node state PrintStyle.set_style(old_style) # reset previous style
static get_style() → Style :
Returns the style, static method
static set_style((Style)arg1) → None :
Set the style, static method
class ecflow.Repeat
Bases:
Boost.Python.instanceRepresents one of RepeatString,RepeatEnumerated,RepeatInteger,RepeatDate,RepeatDay
empty((Repeat)arg1) → bool :
Return true if the repeat is empty.
end((Repeat)arg1) → int :
The last value of the repeat, as an integer
name((Repeat)arg1) → str :
start((Repeat)arg1) → int :
The start value of the repeat, as an integer
step((Repeat)arg1) → int :
The increment for the repeat, as an integer
value((Repeat)arg1) → int :
The current value of the repeat as an integer
class ecflow.RepeatDate
Bases:
Boost.Python.instanceAllows a node to be repeated using a yyyymmdd format
A node can only have one repeat. The repeat name can be referenced in trigger expressions.
Constructor:
RepeatDate(variable,start,end,delta) string variable: The name of the repeat. The current date can referenced in in trigger expressions using the variable name int start: Start date, must have format: yyyymmdd int end: End date, must have format: yyyymmdd int delta<optional>: default = 1, Always in days. The increment used to update the dateException:
- Throws a RuntimeError if start/end are not valid dates
Usage:
rep = RepeatDate("YMD", 20050130, 20050203 ) rep = RepeatDate("YMD", 20050130, 20050203, 2 ) t = Task("t1", RepeatDate("YMD", 20050130, 20050203 ) )end((RepeatDate)arg1) → int :
Return the end date as an integer in yyyymmdd format
name((RepeatDate)arg1) → str :
Return the name of the repeat.
start((RepeatDate)arg1) → int :
Return the start date as an integer in yyyymmdd format
step((RepeatDate)arg1) → int :
Return the step increment. This is used to update the repeat, until end date is reached
class ecflow.RepeatDay
Bases:
Boost.Python.instanceA repeat that is infinite.
A node can only have one repeat.
Constructor:
RepeatDay(step) int step: The step.
Usage:
t = Task("t1", RepeatDay( 1 ))
class ecflow.RepeatEnumerated
Bases:
Boost.Python.instanceAllows a node to be repeated using a enumerated list.
A node can only have one repeat. The repeat can be referenced in trigger expressions.
Constructor:
RepeatEnumerated(variable,list) string variable: The name of the repeat. The current enumeration index can be referenced in trigger expressions using the variable name vector list: The list of enumerationsUsage:
t = Task("t1", RepeatEnumerated("COLOR", [ "red", "green", "blue" ] ))end((RepeatEnumerated)arg1) → int
name((RepeatEnumerated)arg1) → str :
Return the name of the repeat.
start((RepeatEnumerated)arg1) → int
step((RepeatEnumerated)arg1) → int
class ecflow.RepeatInteger
Bases:
Boost.Python.instanceAllows a node to be repeated using a integer range.
A node can only have one repeat. The repeat can be referenced in trigger expressions.
Constructor:
RepeatInteger(variable,start,end,step) string variable: The name of the repeat. The current integer value can be referenced in trigger expressions using the variable name int start: Start integer value int end: End end integer value int step<optional>: Default = 1, The step amountUsage:
t = Task("t1", RepeatInteger("HOUR", 6, 24, 6 ))end((RepeatInteger)arg1) → int
name((RepeatInteger)arg1) → str :
Return the name of the repeat.
start((RepeatInteger)arg1) → int
step((RepeatInteger)arg1) → int
class ecflow.RepeatString
Bases:
Boost.Python.instanceAllows a node to be repeated using a string list.
A node can only have one repeat. The repeat can be referenced in trigger expressions.
Constructor:
RepeatString(variable,list) string variable: The name of the repeat. The current index of the string list can be referenced in trigger expressions using the variable name vector list: The list of enumerationsUsage:
t = Task("t1", RepeatString("COLOR", [ "red", "green", "blue" ] ))end((RepeatString)arg1) → int
name((RepeatString)arg1) → str :
Return the name of the repeat.
start((RepeatString)arg1) → int
step((RepeatString)arg1) → int
class ecflow.SState
Bases:
Boost.Python.enumA SState holds the ecflow_server state
See server states
HALTED = ecflow.SState.HALTED
RUNNING = ecflow.SState.RUNNING
SHUTDOWN = ecflow.SState.SHUTDOWN
names = {'HALTED': ecflow.SState.HALTED, 'RUNNING': ecflow.SState.RUNNING, 'SHUTDOWN': ecflow.SState.SHUTDOWN}
values = {0: ecflow.SState.HALTED, 1: ecflow.SState.SHUTDOWN, 2: ecflow.SState.RUNNING}
class ecflow.State
Bases:
Boost.Python.enumEach node can have a status, which reflects the life cycle of a node.
It varies as follows:
- When the definition file is loaded into the ecflow_server the task status is unknown
- After begin command the task s are either queued, complete, aborted or suspended , a suspended task means that the task is really queued but it must be resumed by the user first before it can be submitted. See
ecflow.DState - Once the dependencies are resolved a task is submitted and placed into the submitted state, however if the submission fails, the task is placed in a aborted state.
- On a successful submission the task is placed into the active state
- Before a job ends, it may send other message to the server such as: Set an event, Change a meter, Change a label, send a message to log file
aborted = ecflow.State.aborted
active = ecflow.State.active
complete = ecflow.State.complete
names = {'complete': ecflow.State.complete, 'unknown': ecflow.State.unknown, 'submitted': ecflow.State.submitted, 'aborted': ecflow.State.aborted, 'active': ecflow.State.active, 'queued': ecflow.State.queued}
queued = ecflow.State.queued
submitted = ecflow.State.submitted
unknown = ecflow.State.unknown
values = {0: ecflow.State.unknown, 1: ecflow.State.complete, 2: ecflow.State.queued, 3: ecflow.State.aborted, 4: ecflow.State.submitted, 5: ecflow.State.active}
class ecflow.Style
Bases:
Boost.Python.enumStyle is used to control printing output for the definition
- DEFS: This style outputs the definition file in a format that is parse-able.
- and can be re-loaded back into the server. Externs are automatically added. This excludes the edit history.
- STATE: The output includes additional state information for debug
- This excludes the edit history
- MIGRATE: Output includes structure and state, allow migration to future ecflow versions
- This includes edit history. If file is reloaded no checking is done
The following shows a summary of the features associated with each choice
Functionality DEFS STATE MIGRATE Auto generate externs Yes Yes No Checking on reload Yes Yes No Edit History No No Yes Show trigger AST No Yes No
DEFS = ecflow.Style.DEFS
MIGRATE = ecflow.Style.MIGRATE
NOTHING = ecflow.Style.NOTHING
STATE = ecflow.Style.STATE
names = {'NOTHING': ecflow.Style.NOTHING, 'DEFS': ecflow.Style.DEFS, 'STATE': ecflow.Style.STATE, 'MIGRATE': ecflow.Style.MIGRATE}
values = {0: ecflow.Style.NOTHING, 1: ecflow.Style.DEFS, 2: ecflow.Style.STATE, 3: ecflow.Style.MIGRATE}
class ecflow.Submittable
Bases:
ecflow.NodeSubmittable is the abstract base class for a Task and Alias
It provides a process id, password and try number
get_aborted_reason((Submittable)arg1) → str :
If node was aborted and a reason was provided, return the string
get_int_try_no((Submittable)arg1) → int :
The current try number as integer.
get_jobs_password((Submittable)arg1) → str :
The password. This generated by server
get_process_or_remote_id((Submittable)arg1) → str :
The process or remote id of the running job
get_try_no((Submittable)arg1) → str :
The current try number as a string.
class ecflow.Suite
Bases:
ecflow.NodeContainerA suite is a collection of Families,Tasks,Variables, repeat and clock definitions
Suite is the only node that can be started using the begin API. There are several ways of adding a suite, see example below and
ecflow.Defs.add_suiteConstructor:
Suite(name, Nodes | attributes) string name : The Suite name. name must consist of alpha numeric characters or underscore or dot. The first character can not be a dot, as this will interfere with trigger expressions. Case is significant Nodes | Attributes:(optional)Exception:
- Throws a RuntimeError if the name is not valid
- Throws a RuntimeError if duplicate suite names added
Usage:
defs = Defs() # create a empty definition. Root of all Suites suite = Suite("suite_1") # create a stand alone suite defs.add_suite(suite) # add suite to definition suite2 = defs.add_suite("s2") # create a suite and add it to the defs defs = Defs( Suite("s1", Family("f1", Task("t1")))) # create in in-placeadd_clock((Suite)arg1, (Clock)arg2) → Suite
add_end_clock((Suite)arg1, (Clock)arg2) → Suite :
End clock, used to mark end of simulation
begun((Suite)arg1) → bool :
Returns true if the suite has begun, false otherwise
get_clock((Suite)arg1) → Clock :
get_end_clock((Suite)arg1) → Clock :
Return the suite’s end clock. Can be NULL
class ecflow.SuiteVec
Bases:
Boost.Python.instanceHold a list of suite nodes’s
append((SuiteVec)arg1, (object)arg2) → None
extend((SuiteVec)arg1, (object)arg2) → None
class ecflow.Task
Bases:
ecflow.SubmittableCreates a task node. Task is a child of a
ecflow.Suiteorecflow.Familynode.Multiple Tasks can be added, however the task names must be unique for a given parent. Note case is significant. Only Tasks can be submitted. A job inside a Task ecf script (i.e.. .ecf file) should generally be re-entrant since a Task may be automatically submitted more than once if it aborts. There are several ways of adding a task, see examples below
Constructor:
Task(name, Attributes) string name : The Task name.Name must consist of alpha numeric characters or underscore or dot. First character can not be a dot. Case is significant attributes: optional, i.e.. like Meter, Event, Trigger etcException:
- Throws a RuntimeError if the name is not valid
- Throws a RuntimeError if a duplicate Task is added
Usage:
task = Task("t1") # create a stand alone task family.add_task(task) # add to the family t2 = family.add_task("t2") # create a task t2 and add to the family # Create Task in place defs = Defs( Suite("s1", Family("f1", Task("t1", Trigger("1==1"), Edit(SLEEP="10"))))) # add Trigger and Variables in place
aliases Returns a list of aliases nodes Returns a list of aliases
class ecflow.TaskVec
Bases:
Boost.Python.instanceHold a list of task nodes
append((TaskVec)arg1, (object)arg2) → None
extend((TaskVec)arg1, (object)arg2) → None
class ecflow.Time
Bases:
Boost.Python.instanceIs used to define a time dependency
This can then control job submission. There can be multiple time dependencies for a node, however overlapping times may cause unexpected results. The time dependency can be made relative to the beginning of the suite or in repeated families relative to the beginning of the repeated family.
Constructor:
Time(string) string: i.e.. "00:30" || "00:30 20:00 00:30" Time(hour,minute,relative<optional> = false) int hour: hour in 24 clock int minute: minute <= 59 bool relative<optional>: default = False, Relative to suite start or repeated node. Time(single,relative<optional> = false) TimeSlot single: A single time bool relative: Relative to suite start or repeated node. Default is false Time(start,finish,increment,relative<optional> = false) TimeSlot start: The start time TimeSlot finish: The finish/end time TimeSlot increment: The increment bool relative<optional>: default = False, relative to suite start or repeated node Time(time_series) TimeSeries time_series:Similar to constructor above
Exceptions:
- raises IndexError when an invalid Time is specified
Usage:
time1 = Time( 10,10 ) # time 10:10 time2 = Time( TimeSlot(10,10), true) # time +10:10 time2 = Time( TimeSlot(10,10), TimeSlot(20,10),TimeSlot(0,10), false ) # time 10:10 20:10 00:10 t = Task("t1", time1,time2,time3, Time("10:30 20:10 00:10")) # Create time in placetime_series((Time)arg1) → TimeSeries :
Return the Time attributes time series
class ecflow.TimeSeries
Bases:
Boost.Python.instanceA TimeSeries can hold a single time slot or a series.
Time series can be created relative to the suite start or start of a repeating node. A Time series can be used as argument to the
ecflow.Time,ecflow.Todayandecflow.Cronattributes of a node. If a time the job takes to complete is longer than the interval, a ‘slot’ is missed e.g time 10:00 20:00 01:00, if the 10.00 run takes more than an hour the 11.00 is missedConstructor:
TimeSeries(single,relative_to_suite_start) TimeSlot single : A single point in a 24 clock optional bool relative_to_suite_start : depend on suite begin time or start of repeating node. Default is false TimeSeries(hour,minute,relative_to_suite_start) int hour : hour in 24 clock int minute : minute < 59 bool relative_to_suite_start<optional> : depend on suite begin time or start of repeating node. Default is false TimeSeries(start,finish,increment,relative_to_suite_start) start TimeSlot : The start time finish TimeSlot : The finish time, when used in a series. This must greater than the start. increment TimeSlot : The increment. This must be less that difference between start and finish bool relative_to_suite_start<optional> : The time is relative suite start, or start of repeating node. The default is falseExceptions:
- Raises IndexError when an invalid time series is specified
Usage:
time_series = TimeSeries(TimeSlot(10,11),False)
finish((TimeSeries)arg1) → TimeSlot :
returns the finish time if time series specified, else returns a NULL time slot
has_increment((TimeSeries)arg1) → bool :
distinguish between a single time slot and a series. returns true for a series
incr((TimeSeries)arg1) → TimeSlot :
returns the increment time if time series specified, else returns a NULL time slot
relative((TimeSeries)arg1) → bool :
returns a boolean where true means that the time series is relative
start((TimeSeries)arg1) → TimeSlot :
returns the start time
class ecflow.TimeSlot
Bases:
Boost.Python.instanceRepresents a time slot.
It is typically used as an argument to a
TimeSeriesor other time dependent attributes of a node.Constructor:
TimeSlot(hour,min) int hour: represent an hour: int minute: represents a minute:
Usage:
ts = TimeSlot(10,11)
empty((TimeSlot)arg1) → bool
hour((TimeSlot)arg1) → int
minute((TimeSlot)arg1) → int
class ecflow.Today
Bases:
Boost.Python.instancetoday is a time dependency that does not wrap to tomorrow.
If the suite s begin time is past the time given for the Today, then the node is free to run.
Constructor:
Today(hour,minute,relative<optional> = false) int hour : hour in 24 clock int minute : minute <= 59 bool relative<optional>: Default = false,Relative to suite start or repeated node. Today(single,relative<optional> = false) TimeSlot single : A single time bool relative : Relative to suite start or repeated node. Default is false Today(start,finish,increment,relative<optional> = false) TimeSlot start : The start time TimeSlot finish : The finish/end time. This must be greater than the start time. TimeSlot increment : The increment bool relative<optional>: Default = false, Relative to suite start or repeated node. Today(time_series) TimeSeries time_series: Similar to constructor above
Exceptions:
- raises IndexError when an invalid Today is specified
Usage:
today1 = Today( 10,10 ) # today 10:10 today2 = Today( TimeSlot(10,10) ) # today 10:10 today3 = Today( TimeSlot(10,10), true) # today +10:10 today4 = Today( TimeSlot(10,10), TimeSlot(20,10),TimeSlot(0,10), false ) # time 10:10 20:10 00:10 t = Task("t1", today1,today2,today3,today4, Today("10:30 20:10 00:10")) # Create today in placetime_series((Today)arg1) → TimeSeries :
Return the Todays time series
class ecflow.Trigger
Bases:
Boost.Python.instanceAdd a trigger or complete expression.
This defines a dependency for a node. There can only be one trigger or complete expression dependency per node. A node with a trigger can only be activated when the trigger has expired. Triggers can reference nodes, events, meters, variables, repeats, limits and late flag A trigger holds a node as long as the expression returns false.
Exception:
- Will throw RuntimeError if first expression is added as ‘AND’ or ‘OR’ expression Like wise second and subsequent expression must have ‘AND’ or ‘OR’ booleans set
Usage:
Multiple trigger will automatically be anded together, If or is required please use bool ‘False’ as the last argument i.e..
task1.add( Trigger("t2 == active" ), Trigger("t1 == complete or t4 == complete" ), Trigger("t5 == active",False))The trigger for task1 is equivalent to
t2 == active and t1 == complete or t4 == complete or t5 == active
Since a large number of triggers are of the form <node> == complete there are are short cuts, these involves a use of a list
task1.add( Trigger( ["t2","t3"] )) # This is same as t2 == complete and t3 == complete
You can also use a node
task1.add( Trigger( ["t2",taskx] ))
If the node ‘taskx’ has a parent, we use the full hierarchy, hence we will get a trigger of the form
t2 ==complete and /suite/family/taskx == complete
If however node taskx has not yet been added to its parent, we use a relative name in the trigger
t2 ==complete and taskx == complete
get_expression((Trigger)arg1) → str :
returns the trigger expression as a string
class ecflow.UrlCmd
Bases:
Boost.Python.instanceExecutes a command ECF_URL_CMD to display a url.
It needs the definition structure and the path to node.
Constructor:
UrlCmd(defs, node_path) defs_ptr defs : pointer to a definition structure string node_path : The node path.
Exceptions
- raises RuntimeError if the definition is empty
- raises RuntimeError if the node path is empty
- raises RuntimeError if the node path can not be found in the definition
- raises RuntimeError if ECF_URL_CMD not defined or if variable substitution fails
Usage: Lets assume that the server has the following definition:
suite s edit ECF_URL_CMD "${BROWSER:=firefox} -remote "openURL(%ECF_URL_BASE%/%ECF_URL%)"" edit ECF_URL_BASE "http://www.ecmwf.int" family f task t1 edit ECF_URL "publications/manuals/ecflow" task t2 edit ECF_URL index.htmltry: ci = Client() ci.sync_local() url = UrlCmd(ci.get_defs(),"/suite/family/task") print(url.execute()) except RuntimeError, e: print(str(e))execute((UrlCmd)arg1) → None :
Displays url in the chosen browser
class ecflow.Variable
Bases:
Boost.Python.instanceDefines a variable on a node for use in ecf script.
A Node can have a number of variables. These variables can be added at any node level: Defs, suite, family or task. The variables are names inside a pair of ‘%’ characters in an ecf script. The content of a variable replaces the variable name in the ecf script at job submission time. When a variable is needed at submission time, it is first sought in the task itself. If it is not found, it is sought from the tasks parent and so on, up through the node levels until found. See variable inheritance A undefined variable in a ecf script, causes the task to be aborted, without the job being submitted.
Constructor:
Variable(name,value) string name: the name of the variable string value: The value of the variable Edit(dict,kwarg) # alternative that allows multiple variables
Usage:
... var = Variable("ECF_JOB_CMD","/bin/sh %ECF_JOB% &") task.add_variable(var) task.add_variable("JOE","90")The following use example of using Edit, which allow multiple variables to added at the same time
t = Task("t1", Edit({ "a":"y", "b":"bb"}, c="v",d="b"), Edit({ "e":1100, "f":"bb"}), Edit(g="d"), Edit(h="1"))defs = Defs( Suite("s1"), Edit(SLEEP="1")) # Add user variable to definition defs.s1 += [ Task("a") ] defs.s1.a += [ Edit({ "x1":"y", "aa1":"bb"}, a="v",b="b"), Edit({ "var":10, "aa":"bb"}), Edit(d="d") ]empty((Variable)arg1) → bool :
Return true if the variable is empty. Used when returning a Null variable, from a find
name((Variable)arg1) → str :
Return the variable name as string
value((Variable)arg1) → str :
Return the variable value as a string
class ecflow.VariableList
Bases:
Boost.Python.instanceHold a list of Variables
append((VariableList)arg1, (object)arg2) → None
extend((VariableList)arg1, (object)arg2) → None
class ecflow.Verify
Bases:
Boost.Python.instance
class ecflow.WhyCmd
Bases:
Boost.Python.instanceThe why command reports, the reason why a node is not running.
It needs the definition structure and the path to node
Constructor:
WhyCmd(defs, node_path) defs_ptr defs : pointer to a definition structure string node_path : The node path
Exceptions:
- raises RuntimeError if the definition is empty
- raises RuntimeError if the node path is empty
- raises RuntimeError if the node path can not be found in the definition
Usage:
try: ci = Client() ci.sync_local() ask = WhyCmd(ci.get_defs(),"/suite/family") print(ask.why()) except RuntimeError, e: print(str(e))why((WhyCmd)arg1) → str :
returns a ‘/n’ separated string, with reasons why node is not running
class ecflow.ZombieAttr
Bases:
Boost.Python.instanceThe zombie attribute defines how a zombie should be handled in an automated fashion
Very careful consideration should be taken before this attribute is added as it may hide a genuine problem. It can be added to any node. But is best defined at the suite or family level. If there is no zombie attribute the default behaviour is to block the init,complete,abort child command. and fob the event,label,and meter child command This attribute allows the server to make a automated response. Please see:
ecflow.ZombieType,ecflow.ChildCmdType,ecflow.ZombieUserActionTypeConstructor:
ZombieAttr(ZombieType,ChildCmdTypes, ZombieUserActionType, lifetime) ZombieType : Must be one of ZombieType.ecf, ZombieType.path, ZombieType.user ChildCmdType : A list(ChildCmdType) of Child commands. Can be left empty in which case the action affect all child commands ZombieUserActionType : One of [ fob, fail, block, remove, adopt ] int lifetime<optional>: Defines the life time in seconds of the zombie in the server. On expiration, zombie is removed automaticallyUsage:
# Add a zombie attribute so that child commands(i.e. ecflow_client --init) # will fail the job if it is a zombie process. s1 = Suite("s1") child_list = [ ChildCmdType.init, ChildCmdType.complete, ChildCmdType.abort ] s1.add_zombie( ZombieAttr(ZombieType.ecf, child_list, ZombieUserActionType.fob)) # create the zombie as part of the node constructor s1 = Suite("s1", ZombieAttr(ZombieType.ecf, child_list, ZombieUserActionType.fail))empty((ZombieAttr)arg1) → bool :
Return true if the attribute is empty
user_action((ZombieAttr)arg1) → ZombieUserActionType :
The automated action to invoke, when zombies arise
zombie_lifetime((ZombieAttr)arg1) → int :
Returns the lifetime in seconds of zombie in the server
zombie_type((ZombieAttr)arg1) → ZombieType :
Returns the zombie type
child_cmds The list of child commands. If empty action applies to all child cmds
class ecflow.ZombieType
Bases:
Boost.Python.enumzombie s are running jobs that fail authentication when communicating with the ecflow_server.
See class zombie type and
ecflow.ZombieAttrfor further information.
ecf = ecflow.ZombieType.ecf
names = {'ecf': ecflow.ZombieType.ecf, 'user': ecflow.ZombieType.user, 'path': ecflow.ZombieType.path}
path = ecflow.ZombieType.path
user = ecflow.ZombieType.user
values = {0: ecflow.ZombieType.user, 1: ecflow.ZombieType.ecf, 2: ecflow.ZombieType.path}
class ecflow.ZombieUserActionType
Bases:
Boost.Python.enumZombieUserActionType is used define an automated response. See class
ZombieAttrThis can be either on the client side or on the server side
client side:
- fob: The child command always succeeds, i.e. allows job to complete without blocking
- fail: The child command is asked to fail.
- block: The child command is asked to block.
- This is the default action for init,complete and abort child commands
server side:
- adopt: Allows the password supplied with the child command s, to be adopted by the server
- kill: Kills the zombie process associated with the child command using ECF_KILL_CMD.
- path zombies will need to be killed manually. If kill is specified for path zombies they will be fobed, i.e. allowed to complete without blocking the job.
- remove: ecflow_server removes the zombie from the zombie list.
- The child continues blocking. If the process is still running, the zombie may well re-appear
Note: Only adopt will allow the child command to continue and change the node tree
adopt = ecflow.ZombieUserActionType.adopt
block = ecflow.ZombieUserActionType.block
fail = ecflow.ZombieUserActionType.fail
fob = ecflow.ZombieUserActionType.fob
kill = ecflow.ZombieUserActionType.kill
names = {'adopt': ecflow.ZombieUserActionType.adopt, 'remove': ecflow.ZombieUserActionType.remove, 'kill': ecflow.ZombieUserActionType.kill, 'fob': ecflow.ZombieUserActionType.fob, 'fail': ecflow.ZombieUserActionType.fail, 'block': ecflow.ZombieUserActionType.block}
remove = ecflow.ZombieUserActionType.remove
values = {0: ecflow.ZombieUserActionType.fob, 1: ecflow.ZombieUserActionType.fail, 2: ecflow.ZombieUserActionType.adopt, 3: ecflow.ZombieUserActionType.remove, 4: ecflow.ZombieUserActionType.block, 5: ecflow.ZombieUserActionType.kill}
ecflow.debug_build() → bool
2 Comments
Andrew Bennett
I've just discovered (at least in v5.7.0) that the automatic addition of `== complete` to a trigger by passing a list of node names only works if the passed node names do not include any path information:
In[6]: import ecflow In[7]: ecflow Out[7]: <module 'ecflow' from '/usr/local/apps/ecflow/5.7.0/lib/python3.8/site-packages/ecflow/__init__.py'> In[8]: print(ecflow.Task('mytask').add( ecflow.Trigger( ["t2","t3"] ))) # as shown in the docs task mytask trigger t2 == complete trigger -a t3 == complete In[9]: print(ecflow.Task('mytask').add( ecflow.Trigger(['../fb/control/archive_prepare']))) # still a list of one node name, but includes path components task mytask trigger ../fb/control/archive_prepare In[10]: print(ecflow.Task('mytask').add( ecflow.Trigger(['../fb/control/archive_prepare','t2']))) # mix, only the node name without the path works task mytask trigger ../fb/control/archive_prepare trigger -a t2 == completeI assume this is intentional behaviour, in which case it should be mentioned in the documentation.
Iain Russell
Hi Andrew, yes, I've checked the code, and it does enforce that the node names should not have any path information. I'm not sure that there is a good reason for that restriction, so we can consider removing it. I'll add it to JIRA in any case and prepare an update to the documentation in the meantime. Thanks for reporting it with a nice example!