| HTML |
|---|
<style type="text/css <div class="section" id="glossary"> .highlight .hll { background-color: #ffffcc } .highlight { background: #eeffcc; } .highlight .c { color: #408090; font-style: italic } /* Comment */ .highlight .err { border: 1px solid #FF0000 } /* Error */ .highlight .k { color: #007020; font-weight: bold } /* Keyword */ .highlight .o { color: #666666 } /* Operator */ .highlight .cm { color: #408090; font-style: italic } /* Comment.Multiline */ .highlight .cp { color: #007020 } /* Comment.Preproc */ .highlight .c1 { color: #408090; font-style: italic } /* Comment.Single */ .highlight .cs { color: #408090; background-color: #fff0f0 } /* Comment.Special */ .highlight .gd { color: #A00000 } /* Generic.Deleted */ .highlight .ge { font-style: italic } /* Generic.Emph */ .highlight .gr { color: #FF0000 } /* Generic.Error */ .highlight .gh { color: #000080; font-weight: bold } /* Generic.Heading */ .highlight .gi { color: #00A000 } /* Generic.Inserted */ .highlight .go { color: #303030 } /* Generic.Output */ .highlight .gp { color: #c65d09; font-weight: bold } /* Generic.Prompt */ .highlight .gs { font-weight: bold } /* Generic.Strong */ .highlight .gu { color: #800080; font-weight: bold } /* Generic.Subheading */ .highlight .gt { color: #0040D0 } /* Generic.Traceback */ .highlight .kc { color: #007020; font-weight: bold } /* Keyword.Constant */ .highlight .kd { color: #007020; font-weight: bold } /* Keyword.Declaration */ .highlight .kn { color: #007020; font-weight: bold } /* Keyword.Namespace */ .highlight .kp { color: #007020 } /* Keyword.Pseudo */ .highlight .kr { color: #007020; font-weight: bold } /* Keyword.Reserved */ .highlight .kt { color: #902000 } /* Keyword.Type */ .highlight .m { color: #208050 } /* Literal.Number */ .highlight .s { color: #4070a0 } /* Literal.String */ .highlight .na { color: #4070a0 } /* Name.Attribute */ .highlight .nb { color: #007020 } /* Name.Builtin */ .highlight .nc { color: #0e84b5; font-weight: bold } /* Name.Class */ .highlight .no { color: #60add5 } /* Name.Constant */ .highlight .nd { color: #555555; font-weight: bold } /* Name.Decorator */ .highlight .ni { color: #d55537; font-weight: bold } /* Name.Entity */ .highlight .ne { color: #007020 } /* Name.Exception */ .highlight .nf { color: #06287e } /* Name.Function */ .highlight .nl { color: #002070; font-weight: bold } /* Name.Label */ .highlight .nn { color: #0e84b5; font-weight: bold } /* Name.Namespace */ .highlight .nt { color: #062873; font-weight: bold } /* Name.Tag */ .highlight .nv { color: #bb60d5 } /* Name.Variable */ .highlight .ow { color: #007020; font-weight: bold } /* Operator.Word */ .highlight .w { color: #bbbbbb } /* Text.Whitespace */ .highlight .mf { color: #208050 } /* Literal.Number.Float */ .highlight .mh { color: #208050 } /* Literal.Number.Hex */ .highlight .mi { color: #208050 } /* Literal.Number.Integer */ .highlight .mo { color: #208050 } /* Literal.Number.Oct */ .highlight .sb { color: #4070a0 } /* Literal.String.Backtick */ .highlight .sc { color: #4070a0 } /* Literal.String.Char */ .highlight .sd { color: #4070a0; font-style: italic } /* Literal.String.Doc */ .highlight .s2 { color: #4070a0 } /* Literal.String.Double */ .highlight .se { color: #4070a0; font-weight: bold } /* Literal.String.Escape */ .highlight .sh { color: #4070a0 } /* Literal.String.Heredoc */ .highlight .si { color: #70a0d0; font-style: italic } /* Literal.String.Interpol */ .highlight .sx { color: #c65d09 } /* Literal.String.Other */ .highlight .sr { color: #235388 } /* Literal.String.Regex */ .highlight .s1 { color: #4070a0 } /* Literal.String.Single */ .highlight .ss { color: #517918 } /* Literal.String.Symbol */ .highlight .bp { color: #007020 } /* Name.Builtin.Pseudo */ .highlight .vc { color: #bb60d5 } /* Name.Variable.Class */ .highlight .vg { color: #bb60d5 } /* Name.Variable.Global */ .highlight .vi { color: #bb60d5 } /* Name.Variable.Instance */ .highlight .il { color: #208050 } /* Literal.Number.Integer.Long */ </style> <div class="section" id="glossary"> <span id="index-0"></span><span id="id1"></span> <dl class="glossary docutils"> <dt id="term-aborted">aborted</dt> <dd><p class="first">Is a<span id="index-0"></span><span id="id1"></span> <dl class="glossary docutils"> <dt id="term-aborted">aborted</dt> <dd><p class="first">Is a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a>.</p> <p class="last">When the ECF_JOB_CMD fails or the <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a> sends a <a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –abort <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a>, then the task is placed into a aborted state.</p> </dd> <dt id="term-active">active</dt> <dd><p class="first">Is a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a>.</p> <p class="last">If <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> was successful, and <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a> has started, then the <a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –init <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a> is received by the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> and the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> is placed into a active state</p> </dd> <dt id="term-autocancel">autocancel</dt> <dd><p class="first">autocancel is a way to automatically delete a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> which has completed.</p> <p>The delete may be delayed by an amount of time in hours and minutes or expressed in days. Any node may have a single autocancel attribute. If the auto cancelled node is referenced in the <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> expression of other nodes it may leave the node waiting. This can be solved by making sure the <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> expression also checks for the <a class="reference internal" href="#term-unknown"><em class="xref std std-term">unknown</em></a> state. i.e.:</p> <div class="highlight-python"><pre>trigger node_to_cancel == complete or node_to_cancel == unknown</pre> </div> <p>This guards against the ‘node_to_cancel’ being undefined or deleted</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Autocancel" title="ecflow.Autocancel"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Autocancel</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_autocancel" title="ecflow.Node.add_autocancel"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_autocancel</span></tt></a>. For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-autocancel"><tt class="xref std std-token docutils literal"><span class="pre">autocancel</span></tt></a></p> </dd> <dt id="term-check-point">check point</dt> <dd><p class="first">The check point file is like the <a class="reference internal" href="#term-suite-definition"><em class="xref std std-term">suite definition</em></a>, but includes all the state information.</p> <p>It is periodically saved by the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a>.</p> <p>It can be used to recover the state of the node tree should server die, or machine crash.</p> <p>By default when a <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> is started it will look to load the check point file.</p> <p class="last">The default check point file name is <host>.<port>.ecf.check. This can be overridden by the ECF_CHECK environment variable</p> </dd> <dt id="term-child-command">child command</dt> <dd><p class="first">Child command’s(or task requests) are called from within the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> files. They include:</p> <blockquote class="last"> <div><table border="1" class="docutils"> <colgroup> <col width="37%" /> <col width="63%" /> </colgroup> <thead valign="bottom"> <tr class="row-odd"><th class="head">Child Command</th> <th class="head">Description</th> </tr> </thead> <tbody valign="top"> <tr class="row-even"><td><a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –init</td> <td>Sets the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> to the <a class="reference internal" href="#term-active"><em class="xref std std-term">active</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a></td> </tr> <tr class="row-odd"><td><a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –event</td> <td>Set an event</td> </tr> <tr class="row-even"><td><a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –meter</td> <td>Change a meter</td> </tr> <tr class="row-odd"><td><a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –label</td> <td>Change a label</td> </tr> <tr class="row-even"><td><a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –wait</td> <td>wait for a expression to evaluate.</td> </tr> <tr class="row-odd"><td><a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> –abort</td> <td>Sets the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> to the <a class="reference internal" href="#term-nodeaborted"><em class="xref std std-term">node<>aborted</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a>.a></td> </p>tr> <p<tr class="last">When the ECF_JOB_CMD fails or the <a row-even"><td><a class="reference internal" href="#term-jobecflow-fileclient"><em class="xref std std-term">job file<>ecflow_client</em></a> sends a –complete</td> <td>Sets the <a class="reference internal" href="#term-ecf-clienttask"><em class="xref std std-term">ecf_client<>task</em></a> –abortto the <a class="reference internal" href="#term-child-commandcomplete"><em class="xref std std-term">child command<>complete</em></a>, then the task is placed into a aborted state.</p><a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a></td> </tr> </tbody> </table> </div></blockquote> </dd> <dt id="term-activeclock">active<>clock</dt> <dd><p class="first">Is>A aclock <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>is an attribute of a <a class="reference internal" href="#term-statussuite"><em class="xref std std-term">status<>suite</em></a>.</p> <p class="last">If <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> was successful, and <a<p>A gain can be specified to offset from the given date.</p> <p>The hyrid and real clock’s always runs in phase with the system clock (UTC in UNIX) but can have any offset from the system clock.</p> <p>The clock can be :</p> <blockquote> <div><ul class="simple"> <li><a class="reference internal" href="#term-jobhybrid-fileclock"><em class="xref std std-term">job>hybrid file<clock</em></a> has started, then the <a a></li> <li><a class="reference internal" href="#term-ecfreal-clientclock"><em class="xref std std-term">ecf_client<>real clock</em></a> –init <a/a></li> <li><a class="reference internal" href="#term-childvirtual-commandclock"><em class="xref std std-term">child>virtual command<clock</em></a> is received by the <a a></li> </ul> </div></blockquote> <p><a class="reference internal" href="#term-ecf-servertime"><em class="xref std std-term">ecf_server<>time</em></a> and, the <a class="reference internal" href="#term-taskday"><em class="xref std std-term">task<>day</em></a> isand placed into a active state</p> </dd> <dt id="term-autocancel">autocancel</dt> <dd>autocancel is a way to automatically delete a <a class="reference internal" href="#term-date"><em class="xref std std-term">date</em></a> <a class="reference internal" href="#term-nodedependencies"><em class="xref std std-term">node<>dependencies</em></a> which has completed. For BNF work a little differently under the two clocks. The default clock type is hybrid.</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/DefinitionecFlow+file+Grammer#grammar-token-autocancelPython+Api#ecflow.Clock" title="ecflow.Clock"><tt class="xref stdpy stdpy-tokenclass docutils literal"><span class="pre">autocancel<>ecflow.Clock</span></tt></a></dd> <dt id="term-child-command">child command</dt> <dd><p class="first">Child command’s(or task requests) are called from within thea> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Suite.add_clock" title="ecflow.Suite.add_clock"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Suite.add_clock</span></tt></a>. For text BNF see <a class="reference internal" href="#term-ecf-script"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-clock"><tt class="xref std std-term">ecf script</em></a> files. They include:</p> <blockquotetoken docutils literal"><span class="pre">clock</span></tt></a></p> </dd> <dt id="term-complete">complete</dt> <dd><p class="last"> <div><p><afirst">Is a <a class="reference internal" href="#term-ecf-clientnode"><em class="xref std std-term">ecf_client<>node</em></a> –init # Sets the <a class="reference internal" href="#term-taskstatus"><em class="xref std std-term">task<>status</em></a>.</p> <p>The node can be set to complete:</p> <blockquote the activeclass="last"> <div><p>By the <a class="reference internal" href="#term-complete-statusexpression"><em class="xref std std-term">status<>complete expression</em></a></p> <p><a<p>At job end when the <a class="reference internal" href="#term-ecf-clienttask"><em class="xref std std-term">ecf_client<>task</em></a> receives –event # Set an event</p> <p><athe <a class="reference internal" href="#term-ecfecflow-client"><em class="xref std std-term">ecf>ecflow_client</em></a> –metercomplete # Change a meter</p> <p><a <a class="reference internal" href="#term-ecfchild-clientcommand"><em class="xref std std-term">ecf_client</em></a> –label # Change a label</p> <p><a>child command</em></a></p> </div></blockquote> </dd> <dt id="term-complete-expression">complete expression</dt> <dd><p class="first">Force a node to be complete <strong>if</strong> the expression evaluates, without running any of the nodes.</p> <p>This allows you to have tasks in the suite which a run only if others fail. In practice the node would need to have a <a class="reference internal" href="#term-ecf-clienttrigger"><em class="xref std std-term">ecf_client<>trigger</em></a> –msg # Send a message to ecFlow-logfile</p> <p><aalso.</p> <p class="last">For python see <a class="reference internal" href="#term-ecf-client"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Expression" title="ecflow.Expression"><tt class="xref stdpy std-term">ecf_client</em><py-class docutils literal"><span class="pre">ecflow.Expression</span></tt></a> –wait # wait for a expression to evaluate</p> <p><a and <a class="reference internal" href="#term-ecf-client"><em class="xref std std-term">ecf_client</em></a> –abort # Sets the/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_complete" title="ecflow.Node.add_complete"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_complete</span></tt></a></p> </dd> <dt id="term-cron">cron</dt> <dd><p class="first">Like time, cron defines time dependency for a <a class="reference internal" href="#term-tasknode"><em class="xref std std-term">task<>node</em></a> to, but it can allow the abort <a class="reference internal" href="#term-statusnode"><em class="xref std std-term">status<>node</em></a><a> to be repeated indefinitely.</p> <p><a<p>When the node becomes complete it will be <a class="reference internal" href="#term-ecf-clientqueued"><em class="xref std std-term">ecf_client<>queued</em></a> –complete # Sets the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> to the completeimmediately. This means that the suite will never complete, and the output is not directly accessible through <a class="reference internal" href="#term-statusecflowview"><em class="xref std std-term">status<>ecflowview</em></a></p> </div></blockquote> </dd> <dt id="term-clock">clock</dt> <dd><p class="first">A clock is an attribute of a <p>If tasks abort, the <a class="reference internal" href="#term-ecflow-suiteserver"><em class="xref std std-term">suite<>ecflow_server</em></a> will not schedule it again.</p> <p>A<p>If clockthe alwaystime runsthe injob phasetakes withto thecomplete systemis clocklonger (UTCthan inthe UNIX)interval but can have any offset from the system clock.</p> <p>The clock must be either hybrid or real:</p> <blockquote> <div><p>Under a hybrid clock, the date never changes unless specifically altered or unless the suite restarts, either automatically or from a begin command.</p> <p>Under a real clock, the date advances by one day at midnight.</p> </div></blockquote> a “slot” is missed, e.g. cron 10:00 20:00 01:00 if the 10:00 run takes more than an hour, the 11:00 run will never occur.</p> <p class="last">Time>For andpython datesee <a class="reference internal" href="#term-dependencies"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Cron" title="ecflow.Cron"><tt class="xref stdpy std-term">dependencies</em></a> work a little differently under the two clocks. The default clock type is hybrid. For py-class docutils literal"><span class="pre">ecflow.Cron</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_cron" title="ecflow.Node.add_cron"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_cron</span></tt></a>. For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammarGrammar#grammar-token-clockcron"><tt class="xref std std-token docutils literal"><span class="pre">clock<>cron</span></tt></a></p> </dd> <dt id="term-completedate">complete<>date</dt> <dd><p class="first">Is>This defines a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a>.</p> <p>The node can be set to complete:</p> <blockquote class="last"> <div><p>By thedate dependency for a node.</p> <p>There can be multiple date dependencies. The European format is used for dates, which is: dd.mm.yy as in 31.12.2007. Any of the three number fields can be expressed with a wildcard <cite>*</cite> to mean any valid value. Thus, 01.*.* means the first day of every month of every year.</p> <p>If a <a class="reference internal" href="#term-completehybrid-triggerclock"><em class="xref std std-term">complete>hybrid trigger<clock</em></a></p> <p>At job end when the a> is defined, any node held by a date dependency will be set to <a class="reference internal" href="#term-taskcomplete"><em class="xref std std-term">task<>complete</em></a> receivesat the beginning of the <a class="reference internal" href="#term-ecf-clientsuite"><em class="xref std std-term">ecf_client<>suite</em></a> –complete, without running the corresponding job. Otherwise under a hybrid clock the <a class="reference internal" href="#term-child-commandsuite"><em class="xref std std-term">child command<>suite</em></a></p> </div></blockquote> </dd> <dt id="term-complete-trigger">complete trigger</dt> <dd><pa> would never <a class="reference internal" href="#term-complete"><em class="first">Forcexref a node to be complete <strong>if</strong> the expression evaluates, without running any of the nodesstd std-term">complete</em></a>.</p> <p class="last">This>For allows you to have tasks in the suite which a run only if others fail. In practice the node would need to have apython see: <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> also.</p> </dd> <dt id="term-cron">cron</dt> <dd>Like time, cron defines time dependency for a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>, but it can allow the /wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Date" title="ecflow.Date"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Date</span></tt></a> and <a class="reference internal" href="#term-node"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_date" title="ecflow.Node.add_date"><tt class="xref stdpy std-term">node</em></a> to be repeated indefinitely. Forpy-class docutils literal"><span class="pre">ecflow.Node.add_date</span></tt></a>. For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammarGrammar#grammar-token-crondate"><tt class="xref std std-token docutils literal"><span class="pre">cron<>date</span></tt></a></p> </dd> <dt id="term-dateday">date<>day</dt> <dd><p class="first">This defines a dateday dependency for a node.</p> <p class="last">There<p>There can be multiple dateday dependencies. The European format is used for dates, which is: dd.mm.yy as in 31.12.2007. Any of the three number fields can be expressed with a wildcard <cite>*</cite> to mean any valid value. Thus, 01.*.* means the first day of every month of every year. For BNF see</p> <p>If a <a class="reference internal" href="#term-hybrid-clock"><em class="xref std std-term">hybrid clock</em></a> is defined, any node held by a day dependency will be set to <a class="reference internal" href="#term-complete"><em class="xref std std-term">complete</em></a> at the beginning of the <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a>, without running the corresponding job. Otherwise under a hybrid clock the <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-date"><tt#term-suite"><em class="xref std std-token docutils literal"><spanterm">suite</em></a> would never <a class="pre">date</span></tt></a></p> </dd> <dt id="term-day">day</dt> <dd><preference internal" href="#term-complete"><em class="first">Thisxref defines a day dependency for a nodestd std-term">complete</em></a>.</p> <p class="last">There>For can be multiple day dependencies. For BNF seepython see: <a class="reference internal" href="/wiki/display/ECFLOW/DefinitionecFlow+file+Grammer#grammar-token-dayPython+Api#ecflow.Day" title="ecflow.Day"><tt class="xref stdpy stdpy-tokenclass docutils literal"><span class="pre">day<>ecflow.Day</span></tt></a></p> </dd> <dt id="term-defstatus">defstatus</dt> <dd><p class="first">Defines the default status for a task/family to be assigned to the a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_day" title="ecflow.Node.add_day"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_day</span></tt></a>. For text BNF see <a class="reference internal" href="#term-node"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-day"><tt class="xref std std-term">node</em></a> when the begin command is issued.</p> <ptoken docutils literal"><span class="pre">day</span></tt></a></p> </dd> <dt id="term-defstatus">defstatus</dt> <dd><p class="lastfirst">By>Defines the default <a class="reference internal" href="#term-nodestatus"><em class="xref std std-term">node<>status</em></a> getsfor queued when a task/family to be assigned to the <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> when the begin command is issued.</p> <p>By default <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> gets queued when you use begin on a <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a>. defstatus is useful in preventing suites from running automatically once begun or in setting tasks complete so they can be run selectively. For BNF</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/DefinitionecFlow+file+Grammer#grammar-token-defstatusPython+Api#ecflow.DState" title="ecflow.DState"><tt class="xref stdpy stdpy-tokenclass docutils literal"><span class="pre">defstatus<>ecflow.DState</span></tt></a></p> </dd> <dt id="term-dependencies">dependencies</dt> <dd><p class="first">Dependencies are attributes of node. They include a> and <a class="reference internal" href="#term-trigger"><em="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_defstatus" title="ecflow.Node.add_defstatus"><tt class="xref stdpy std-term">trigger</em></a>, <a class="reference internal" href="#term-date"><empy-class docutils literal"><span class="xref std std-term">date</em></a>,pre">ecflow.Node.add_defstatus</span></tt></a>. For text BNF see <a class="reference internal" href="#term-day"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-defstatus"><tt class="xref std std-term">day</em></a>, <a class="token docutils literal"><span class="pre">defstatus</span></tt></a></p> </dd> <dt id="term-dependencies">dependencies</dt> <dd><p class="first">Dependencies are attributes of node, that can suppress/hold a <a class="reference internal" href="#term-timetask"><em class="xref std std-term">time<>task</em></a> from taking part in <a class="reference internal" href="#term-job-todaycreation"><em class="xref std std-term">today<>job creation</em></a>,.</p> <p>They include <a class="reference internal" href="#term-crontrigger"><em class="xref std std-term">cron<>trigger</em></a>, <a class="reference internal" href="#term-complete-triggerdate"><em class="xref std std-term">complete trigger<>date</em></a>, <a class="reference internal" href="#term-inlimitday"><em class="xref std std-term">inlimit<>day</em></a>, and <a class="reference internal" href="#term-limittime"><em class="xref std std-term">limit<>time</em></a>.</p> <p class="last">A , <a class="reference internal" href="#term-nodetoday"><em class="xref std std-term">node<>today</em></a>, that is dependent can not be started as long as some dependency is holding it.</p> </dd> <dt id="term-directives">directives</dt> <dd><p class="first">directives are expanded during <a class="reference internal" href="#term-cron"><em class="xref std std-term">cron</em></a>, <a class="reference internal" href="#term-complete-expression"><em class="xref std std-term">complete expression</em></a>, <a class="reference internal" href="#term-inlimit"><em class="xref std std-term">inlimit</em></a> and <a class="reference internal" href="#term-pre-processinglimit"><em class="xref std std-term">pre-processing<>limit</em></a>. Examples include:</p> <blockquote<p>A <a class="last"> <div><p>%include <filename></p> <p>%comment : start’s a comment, which is ended by %end directive. The section enclosed by %comment - %end is removed during :term:` pre-processing`</p> <p>%manual : start’s a manual, which is ended by %end directive. The section enclosed by %manual - %end is removed during :term:` pre-processing` However the manual directive is used to create thereference internal" href="#term-task"><em class="xref std std-term">task</em></a> that is dependent can not be started as long as some dependency is holding it or any of its parent <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> s.</p> <p class="last">The <a class="reference internal" href="#term-manualecflow-pageserver"><em class="xref std std-term">manual page<>ecflow_server</em></a></p> <p>%noppa> will :check stopsthe pre-processingdependencies until a line stating with %end is found</p> <p>%end : End pre-processing of %comment, %manual or %nopp</p> <p>%VAR% : This direct’s the server to perform every minute, during normal <a class="reference internal" href="#term-variable-substitutionscheduling"><em class="xref std std-term">variable substitution<>scheduling</em></a>. This involves searching for a <strong>and</strong> when any <a class="reference internal" href="#term-suitechild-definitioncommand"><em class="xref std std-term">suite>child definition<command</em></a> causes a state change in the <a class="reference internal" href="#term-variablesuite-definition"><em class="xref std std-term">variable<>suite definition</em></a> or generated <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a> of name VAR and substituting in the value of the variable.</p> </div></blockquote> </dd> <dt id="term-ecf-script">ecf script</dt> <dd><p class="first">The ecFlow script refers to an ‘.ecf’ file.</p> <p class="last">This is similar to a UNIX shell script. The differences, however, includes the addition of “C” like pre-processing.</p> </dd> <dt id="term-directives">directives</dt> <dd><p class="first">Directives start with a % charater. This is referred to as <a class="reference internal" href="#term-ecf-micro"><em class="xref std std-term">ECF_MICRO</em></a> character. These directives start as the first character on a line in a <a class="reference internal" href="#term-ecf-directivesscript"><em class="xref std std-term">directives<>ecf script</em></a> and ecFlow <a file. The exception is where we use two ECF_MICRO. ie %VAR%, in this case they can occur anywhere on the line. (These directives take part in <a class="reference internal" href="#term-variable-substitution"><em class="xref std std-term">variable<>variable substitution</em></a>‘s.</p> </dd> <dt id="term-ecf-client">ecf_client</dt> <dd><p class="first">This executable is a command line program; it is used for all communication with the server.</p> <p>To see the full range of commands that can be sent to the .) Directives are expanded during <a class="reference internal" href="#term-ecfpre-serverprocessing"><em class="xref std std-term">ecf_server<>pre-processing</em></a>. type the following in a UNIX shellExamples include:</p> <blockquote> <div>ecf_client –help</div></blockquote> <p class="last">This functionality is also provided by the <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#python-api"><em>ecFlow Python Api</em></a> see class <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Client" title="ecflow.Client"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Client</span></tt></a></p> </dd> <dt id="term-ecf-server">ecf_server</dt> <dd><p class="first">This executable is the server.</p> <p>It is responsible for <a class="reference internal" href="#term-scheduling"><em class="xref std std-term">scheduling</em></a> the jobs and responding to <a class="reference internal" href="#term-ecf-client"><em class="xref std std-term">ecf_client</em></a> requests</p> <p>Multiple servers can be run on the same machine/host providing they are assigned a unique port number.</p> <p>The server record’s all request’s in the log file.</p> <p>The server will periodically write out a check point file.</p> <p class="last">A check point file is the <a class="reference internal" href="#term-suite-definition"><em class="xref std std-term">suite definition</em></a> with additional state information.</p> </dd> <dt id="term-ecflow">ecFlow</dt> <dd>ecFlow is the Supervisor Monitoring Scheduler software in place at ECMWF that helps computer jobs design, submission and monitoring both in the research and the operations departments.</dd> <dt id="term-ecflowview">ecFlowview</dt> <dd><p class="first">ecFlowview executable is the GUI based client, that is used to visualise and monitor</p> <blockquote class="last"> <div><p>The hierarchical structure of the <table border="1" class="last docutils"> <colgroup> <col width="22%" /> <col width="78%" /> </colgroup> <thead valign="bottom"> <tr class="row-odd"><th class="head">Symbol</th> <th class="head">Meaning</th> </tr> </thead> <tbody valign="top"> <tr class="row-even"><td>%include <filename></td> <td>%ECF_INCLUDE% directory is searched for the <tt class="file docutils literal"><span class="pre">filename</span></tt> and the contents included into the job file. If that variable is not defined ECF_HOME is used. This is the recommended format</td> </tr> <tr class="row-odd"><td>%include “filename”</td> <td>Include the contents of the file: %ECF_HOME%/%SUITE%/%FAMILY%/filename into the job.</td> </tr> <tr class="row-even"><td>%include filename</td> <td>Include the contents of the file <tt class="file docutils literal"><span class="pre">filename</span></tt> into the output. The only form that can be used safely must start with a slash ‘/’</td> </tr> <tr class="row-odd"><td>%includenopp filename</td> <td>Same as %include, but the file is not interpreted at all.</td> </tr> <tr class="row-even"><td>%comment</td> <td>Start’s a comment, which is ended by %end directive. The section enclosed by %comment - %end is removed during <a class="reference internal" href="#term-suitepre-definitionprocessing"><em class="xref std std-term">suite definition<>pre-processing</em></a></p> <p>state changes in the/td> </tr> <tr class="row-odd"><td>%manual</td> <td>Start’s a manual, which is ended by %end directive. The section enclosed by %manual - %end is removed during <a class="reference internal" href="#term-nodepre-processing"><em class="xref std std-term">node<>pre-processing</em></a>‘s and. The manual directive is used to create the <a class="reference internal" href="#term-ecfmanual-serverpage"><em class="xref std std-term">ecf_server<>manual page</em></a>, using colour coding</p> <p>Attributes of the nodes and any show in <a class="reference internal" href="#term-dependenciesecflowview"><em class="xref std std-term">dependencies<>ecflowview</em></a></p> <p><a/a>.</td> </tr> <tr class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> file and the corresponding <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a></p> </div></blockquote> </dd> <dt id="term-event">event</dt> <dd><p class="first">The purpose of an event is to signal partial completion of a <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> and to be able to trigger another job which is waiting for this partial completion.</p> <p>Only tasks can have events and they can be considered as an attribute of arow-even"><td>%nopp</td> <td>Stop pre-processing until a line starting with %end is found. No interpretation of the text will be done( i.e. no variable substitutions)</td> </tr> <tr class="row-odd"><td>%end</td> <td>End processing of %comment or %manual or %nopp</td> </tr> <tr class="row-even"><td>%ecfmicro CHAR</td> <td>Change the directive character, to the character given. If set in an include file the effect is retained for the rest of the job( or until set again). It should be noted that the ecfmicro directive specified in the <a class="reference internal" href="#term-taskecf-script"><em class="xref std std-term">task<>ecf script</em></a>.</p> <p>There can be many events and they are displayed as nodes.</p> <p class="last">An event has a number and possibly a name. If it is only defined as a number, its name is the text representation of the number without leading zeroes. For BNF see file, does <strong>not</strong> effect the variable substitution for ECF_JOB_CMD, ECF_KILL_CMD or ECF_STATUS_CMD variables. They still use <a class="reference internal" href="#term-ecf-micro"><em class="xref std std-term">ECF_MICRO</em></a>. If no ecfmicro directive exists, we default to using <a class="reference internal" href="#term-ecf-micro"><em class="xref std std-term">ECF_MICRO</em></a> from the <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-event"><tt#term-suite-definition"><em class="xref std std-token docutils literal"><span class="pre">event</span></tt></a></p>term">suite definition</em></a></td> </tr> </tbody> </table> </dd> <dt id="term-extern">extern<-ecf-file-location-algorithm">ecf file location algorithm</dt> <dd><p class="first">This allows an external <a ><a class="reference internal" href="#term-nodeecflow-server"><em class="xref std std-term">node<>ecflow_server</em></a> and job creation checking uses the following algorithm to be used inlocate the ‘.ecf’ file corresponding to a <a class="reference internal" href="#term-triggertask"><em class="xref std std-term">trigger<>task</em></a> expression.:</p> <p>All <a<ul> <li><p class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>‘s in <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a>‘s must be known to <a class="reference internal" href="#term-ecf-server"><em class="xref std std-term">ecf_server</em></a> by the end of the load command. No cross-suite <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a> are allowed unless the names of tasks outside the suite are declared as external. An external <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> reference is considered unknown if it is not defined when the <a class="reference internalfirst">ECF_SCRIPT</p> <p>First it uses the generated variable ECF_SCRIPT to locate the script. This variable is generated from: ECF_HOME/<path to task>.ecf</p> <p>Hence if the task path is /suite/f1/f2/t1, then ECF_SCRIPT=ECF_HOME/suite/f1/f2/t1.ecf</p> </li> <li><p class="first">ECF_FILES</p> <p>Second it checks for the user defined ECF_FILES variable. If defined the value of this variable must correspond to a directory. This directory is searched in reverse order.</p> <p>i.e lets assume we have a task: /o/12/fc/model and ECF_FILES is defined as: /home/ecmwf/emos/def/o/ECFfiles</p> <p>The ecFlow will use the following search pattern.</p> <blockquote> <div><ol class="arabic simple"> <li>/home/ecmwf/emos/def/o/ECFfiles/o/12/fc/model.ecf</li> <li>/home/ecmwf/emos/def/o/ECFfiles/12/fc/model.ecf</li> <li>/home/ecmwf/emos/def/o/ECFfiles/fc/model.ecf</li> <li>/home/ecmwf/emos/def/o/ECFfiles/model.ecf</li> </ol> </div></blockquote> </li> <li><p class="first">ECF_HOME</p> <p>Thirdly it searchs for the script in reverse order using ECF_HOME (i.e like ECF_FILES)</p> </li> </ul> <blockquote class="last"> <div><p>If this fails, than the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> is placed into the <a class="reference internal" href="#term-aborted"><em class="xref std std-term">aborted</em></a> state. We can check that file can be located before loading the suites into the server.</p> <blockquote> <div><ul class="simple"> <li><a class="reference internal" href="/wiki/display/ECFLOW/Checking+job+creation#checking-job-creation"><em>Checking job creation</em></a></li> <li><a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Defs.check_job_creation" title="ecflow.Defs.check_job_creation"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Defs.check_job_creation</span></tt></a></li> </ul> </div></blockquote> </div></blockquote> </dd> <dt id="term-ecf-script">ecf script</dt> <dd><p class="first">The ecFlow script refers to an ‘.ecf’ file.</p> <p>The script file is transformed into the <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a> by the <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> process.</p> <p>The base name of the script file <strong>must</strong> match its corresponding <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>. i.e t1.ecf , corresponds to the task of name ‘t1’. The script if placed in the ECF_FILES directory, may be re-used by multiple tasks belonging to different families, providing the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> name matches.</p> <p>The ecFlow script is similar to a UNIX shell script.</p> <p class="last">The differences, however, includes the addition of “C” like pre-processing <a class="reference internal" href="#term-directives"><em class="xref std std-term">directives</em></a> and ecFlow <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a>‘s. Also the script <em>must</em> include calls to the <strong>init</strong> and <strong>complete</strong> <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a> s so that the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> is aware when the job starts(i.e changes state to <a class="reference internal" href="#term-active"><em class="xref std std-term">active</em></a>) and finishes ( i.e changes state to <a class="reference internal" href="#term-complete"><em class="xref std std-term">complete</em></a>)</p> </dd> <dt id="term-ecf-micro">ECF_MICRO</dt> <dd>This is a generated <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a>. The default value is %. This variable is used in <a class="reference internal" href="#term-variable-substitution"><em class="xref std std-term">variable substitution</em></a> during command invocation and default directive character during <a class="reference internal" href="#term-pre-processing"><em class="xref std std-term">pre-processing</em></a>. It can be overriden, but must be replaced by a single character.</dd> <dt id="term-ecf-tryno">ECF_TRYNO</dt> <dd><p class="first">This is a generated <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a> that is used in file name generation. It represents the current try number for the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>.</p> <p class="last">After <strong>begin</strong> it is set to 1. The number is advanced if the job is re-run. It is re-set back to 1 after a re-queue. It is used in output and <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a> numbering. (i.e It avoids overwriting the <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a> output during multiple re-runs)</p> </dd> <dt id="term-ecflow">ecFlow</dt> <dd>ecFlow is the Supervisor Monitoring Scheduler software in place at ECMWF that helps computer jobs design, submission and monitoring both in the research and the operations departments.</dd> <dt id="term-ecflow-client">ecflow_client</dt> <dd><p class="first">This executable is a command line program; it is used for all communication with the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a>.</p> <p>To see the full range of commands that can be sent to the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> type the following in a UNIX shell:</p> <blockquote> <div>ecflow_client –help</div></blockquote> <p>This functionality is also provided by the <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#client-server-python-api"><em>client server python API</em></a>.</p> <p>The following variables affect the execution of ecflow_client.</p> <p>Since the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> can call ecflow_client then typically some are set in an include header. i.e. <a class="reference internal" href="/wiki/display/ECFLOW/Understanding+Includes#head-h"><em>head.h</em></a>.</p> <table border="1" class="last docutils"> <colgroup> <col width="12%" /> <col width="49%" /> <col width="8%" /> <col width="31%" /> </colgroup> <thead valign="bottom"> <tr class="row-odd"><th class="head">Variable Name</th> <th class="head">Explanation</th> <th class="head">Compulsory</th> <th class="head">Example</th> </tr> </thead> <tbody valign="top"> <tr class="row-even"><td>ECF_NODE</td> <td>Name of the host running the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a></td> <td>Yes</td> <td>oetzi</td> </tr> <tr class="row-odd"><td>ECF_NAME</td> <td>Path to the task</td> <td>Yes</td> <td>/suite/family/task</td> </tr> <tr class="row-even"><td>ECF_PASS</td> <td>Jobs password.</td> <td>Yes</td> <td>(generated)</td> </tr> <tr class="row-odd"><td>ECF_RID</td> <td>Remote id. Allow easier job kill, and disambiguate a zombie from the real job.</td> <td>Yes</td> <td>(generated)</td> </tr> <tr class="row-even"><td>ECF_PORT</td> <td>Port number of the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a></td> <td>No</td> <td>3141.Must match <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a></td> </tr> <tr class="row-odd"><td>ECF_TRYNO</td> <td>Task try number.</td> <td>No</td> <td>(generated)</td> </tr> <tr class="row-even"><td>ECF_HOSTFILE</td> <td>File to list alternate servers</td> <td>No</td> <td>/home/user/avi/.ecfhostfile</td> </tr> <tr class="row-odd"><td>ECF_TIMEOUT</td> <td>Maximum time is seconds for the client to deliver message</td> <td>No</td> <td>24*3600</td> </tr> <tr class="row-even"><td>ECF_DENIED</td> <td>If server denies client communication and this flag is set, exit with an error. Avoids 24hr hour connection attempt to <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a>.</td> <td>No</td> <td>1</td> </tr> </tbody> </table> </dd> <dt id="term-ecflow-server">ecflow_server</dt> <dd><p class="first">This executable is the server.</p> <p>It is responsible for <a class="reference internal" href="#term-scheduling"><em class="xref std std-term">scheduling</em></a> the jobs and responding to <a class="reference internal" href="#term-ecflow-client"><em class="xref std std-term">ecflow_client</em></a> requests</p> <p>Multiple servers can be run on the same machine/host providing they are assigned a unique port number.</p> <p>The server record’s all request’s in the log file.</p> <p>The server will periodically(See ECF_CHECKINTERVAL) write out a <a class="reference internal" href="#term-check-point"><em class="xref std std-term">check point</em></a> file.</p> <p>The following environment variables control the execution of the server and may be set before the start of the server. ecflow_server will start happily with out any of these variables being set, since all of them have default values.</p> <table border="1" class="docutils"> <colgroup> <col width="15%" /> <col width="51%" /> <col width="34%" /> </colgroup> <thead valign="bottom"> <tr class="row-odd"><th class="head">Variable Name</th> <th class="head">Explanation</th> <th class="head">Default value</th> </tr> </thead> <tbody valign="top"> <tr class="row-even"><td>ECF_HOME</td> <td>Home for all the ECF files</td> <td>Current working directory</td> </tr> <tr class="row-odd"><td>ECF_PORT</td> <td>Server port number. Must be unique</td> <td>3141</td> </tr> <tr class="row-even"><td>ECF_LOG</td> <td>History or log file</td> <td><host>.<port>.ecf.log</td> </tr> <tr class="row-odd"><td>ECF_CHECK</td> <td>Name of the checkpoint file</td> <td><host>.<port>.ecf.check</td> </tr> <tr class="row-even"><td>ECF_CHECKOLD</td> <td>Name of the backup checkpoint file</td> <td><host>.<port>.ecf.check.b</td> </tr> <tr class="row-odd"><td>ECF_CHECKINTERVAL</td> <td>Interval in second to save <a class="reference internal" href="#term-check-point"><em class="xref std std-term">check point</em></a> file</td> <td>120</td> </tr> <tr class="row-even"><td>ECF_LISTS</td> <td>White list file. Controls read/write access to the server for each user</td> <td><host>.<port>.ecf.lists</td> </tr> </tbody> </table> <p class="last">The server can be in several states. The default when first started is <a class="reference internal" href="#term-halted"><em class="xref std std-term">halted</em></a>, See <a class="reference internal" href="#term-server-states"><em class="xref std std-term">server states</em></a></p> </dd> <dt id="term-ecflowview">ecflowview</dt> <dd><p class="first">ecflowview executable is the GUI based client, that is used to visualise and monitor the hierarchical structure of the <a class="reference internal" href="#term-suite-definition"><em class="xref std std-term">suite definition</em></a></p> <blockquote class="last"> <div><p>state changes in the <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>‘s and the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a>, using colour coding</p> <p>Attributes of the nodes and any <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a></p> <p><a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> file and the corresponding <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a></p> </div></blockquote> </dd> <dt id="term-event">event</dt> <dd><p class="first">The purpose of an event is to signal partial completion of a <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> and to be able to trigger another job which is waiting for this partial completion.</p> <p>Only tasks can have events and they can be considered as an attribute of a <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>.</p> <p>There can be many events and they are displayed as nodes.</p> <p>The event is updated by placing the –event <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a> in a <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a>.</p> <p>An event has a number and possibly a name. If it is only defined as a number, its name is the text representation of the number without leading zeroes.</p> <p>For python see: <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Event" title="ecflow.Event"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Event</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_event" title="ecflow.Node.add_event"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_event</span></tt></a> For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-event"><tt class="xref std std-token docutils literal"><span class="pre">event</span></tt></a></p> <p class="last">Events can be referenced in <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> and <a class="reference internal" href="#term-complete-expression"><em class="xref std std-term">complete expression</em></a> s.</p> </dd> <dt id="term-extern">extern</dt> <dd><p class="first">This allows an external <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> to be used in a <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> expression.</p> <p>All <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>‘s in <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a>‘s must be known to <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> by the end of the load command. No cross-suite <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a> are allowed unless the names of tasks outside the suite are declared as external. An external <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> reference is considered unknown if it is not defined when the <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> is evaluated. You are strongly advised to avoid cross-suite <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a>.</p> <p class="last">Families and suites that depend on one another should be placed in a single <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a>. If you think you need cross-suite dependencies, you should consider merging the suites together and have each as a top-level family in the merged suite. For BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-extern"><tt class="xref std std-token docutils literal"><span class="pre">extern</span></tt></a></p> </dd> <dt id="term-family">family</dt> <dd><p class="first">A family is an organisational entity that is used to provide hierarchy and grouping. It consists of a collection of <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>‘s and families.</p> <p>Typically you place tasks that are related to each other inside the same family, analogous to the way you create directories to contain related files. For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Family" title="ecflow.Family"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Family</span></tt></a>. For BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-family"><tt class="xref std std-token docutils literal"><span class="pre">family</span></tt></a></p> <p class="last">It serves as an intermediate <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> in a <a class="reference internal" href="#term-suite-definition"><em class="xref std std-term">suite definition</em></a>.</p> </dd> <dt id="term-halted">halted</dt> <dd>Is a <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> state. See <a class="reference internal" href="#term-server-states"><em class="xref std std-term">server states</em></a></dd> <dt id="term-hybrid-clock">hybrid clock</dt> <dd><p class="first">A hybrid <a class="reference internal" href="#term-clock"><em class="xref std std-term">clock</em></a> is a complex notion: the date and time are not connected.</p> <p>The date has a fixed value during the complete execution of the <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a>. This will be mainly used in cases where the suite does not <a class="reference internal" href="#term-complete"><em class="xref std std-term">complete</em></a> in less than 24 hours. This guarantees that all tasks of this suite are using the same <a class="reference internal" href="#term-date"><em class="xref std std-term">date</em></a>. On the other hand, the time follows the time of the machine.</p> <p>Hence the <a class="reference internal" href="#term-date"><em class="xref std std-term">date</em></a> never changes unless specifically altered or unless the suite restarts, either automatically or from a begin command.</p> <p class="last">Under a hybrid <a class="reference internal" href="#term-clock"><em class="xref std std-term">clock</em></a> any <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> held by a <a class="reference internal" href="#term-date"><em class="xref std std-term">date</em></a> or <a class="reference internal" href="#term-day"><em class="xref std std-term">day</em></a> dependency will be set to complete at the beginning of the suite. (i.e without its job ever running). Otherwise the <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a> would never <a class="reference internal" href="#term-complete"><em class="xref std std-term">complete</em></a>.</p> </dd> <dt id="term-inlimit">inlimit</dt> <dd><p class="first">The inlimit works in conjunction with <a class="reference internal" href="#term-limit"><em class="xref std std-term">limit</em></a>/<a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Limit" title="ecflow.Limit"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Limit</span></tt></a> for providing simple load management</p> <p>inlimit is added to the <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> that needs to be limited.</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.InLimit" title="ecflow.InLimit"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.InLimit</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_inlimit" title="ecflow.Node.add_inlimit"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_inlimit</span></tt></a>. For text BNF see <a class="reference internal" href="#term-inlimit"><em class="xref std std-term">inlimit</em></a></p> </dd> <dt id="term-job-creation">job creation</dt> <dd><p class="first">Job creation or task invocation can be initiated manually via <a class="reference internal" href="#term-ecflowview"><em class="xref std std-term">ecflowview</em></a> but also by the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> during <a class="reference internal" href="#term-scheduling"><em class="xref std std-term">scheduling</em></a> when a <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> (and <em>all</em> of its parent <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> s) is free of its <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a>.</p> <p>The process of job creation includes:</p> <blockquote> <div><p>o Generating a unique password ECF_PASS, which is placed in <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> during <a class="reference internal" href="#term-pre-processing"><em class="xref std std-term">pre-processing</em></a>. See <a class="reference internal" href="/wiki/display/ECFLOW/Understanding+Includes#head-h"><em>head.h</em></a></p> <p>o Locating <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> files , corresponding to the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> in the <a class="reference internal" href="#term-suite-definition"><em class="xref std std-term">suite definition</em></a>, See <a class="reference internal" href="#term-ecf-file-location-algorithm"><em class="xref std std-term">ecf file location algorithm</em></a></p> <p>o <a class="reference internal" href="#term-pre-processing"><em class="xref std std-term">pre-processing</em></a> the contents of the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> file</p> </div></blockquote> <p>The steps above transforms an <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> to a <a class="reference internal" href="#term-job-file"><em class="xref std std-term">job file</em></a> that can be submitted by performing <a class="reference internal" href="#term-variable-substitution"><em class="xref std std-term">variable substitution</em></a> on the ECF_JOB_CMD <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a> and invoking the command.</p> <p>The running jobs will communicate back to the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> by calling <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a>‘s.</p> <p class="last">This causes <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a> changes on the <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>‘s in the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> and flags can be set to indicate various events.</p> </dd> <dt id="term-job-file">job file</dt> <dd><p class="first">The job file is created by the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> during <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> using the <a class="reference internal" href="#term-ecf-tryno"><em class="xref std std-term">ECF_TRYNO</em></a> <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a></p> <p>It is derived from the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> after expanding the pre-processing <a class="reference internal" href="#term-directives"><em class="xref std std-term">directives</em></a>.</p> <p>It has the form <task name>.job<<a class="reference internal" href="#term-ecf-tryno"><em class="xref std std-term">ECF_TRYNO</em></a>>”, i.e. t1.job1.</p> <p>Note job creation checking will create a job file with an extension with zero. i.e ‘.job0’. See <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Defs.check_job_creation" title="ecflow.Defs.check_job_creation"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Defs.check_job_creation</span></tt></a></p> <p class="last">When the job is run the output file has the <a class="reference internal" href="#term-ecf-tryno"><em class="xref std std-term">ECF_TRYNO</em></a> as the extension. i.e t1.1 where ‘t1’ represents the task name and ‘1’ the <a class="reference internal" href="#term-ecf-tryno"><em class="xref std std-term">ECF_TRYNO</em></a></p> </dd> <dt id="term-label">label</dt> <dd><p class="first">A label has a name and a value and is a way of <strong>displaying</strong> information in <a class="reference internal" href="#term-ecflowview"><em class="xref std std-term">ecflowview</em></a></p> <p>By placing a label <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a> s in the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> the user can be informed about progress in <a class="reference internal" href="#term-ecflowview"><em class="xref std std-term">ecflowview</em></a>.</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Label" title="ecflow.Label"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Label</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_label" title="ecflow.Node.add_label"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_label</span></tt></a>. For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-label"><tt class="xref std std-token docutils literal"><span class="pre">label</span></tt></a></p> </dd> <dt id="term-late">late</dt> <dd><p class="first">Define a tag for a node to be late.</p> <p>Suites cannot be late, but you can define a late tag for submitted in a suite, to be inherited by the families and tasks. When a node is classified as being late, the only action <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> takes is to set a flag. <a class="reference internal" href="#term-ecflowview"><em class="xref std std-term">ecflowview</em></a> will display these alongside the <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> name as an icon (and optionally pop up a window).</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Late" title="ecflow.Late"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Late</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_late" title="ecflow.Node.add_late"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_late</span></tt></a>. For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-late"><tt class="xref std std-token docutils literal"><span class="pre">late</span></tt></a></p> </dd> <dt id="term-limit">limit</dt> <dd><p class="first">limit provides a means of providing simple load management by say limiting the number of tasks submitted to a specific server. Typically you either define limits on <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a> level or define a separate suite to hold limits so that they can be used by multiple suites.</p> <p>The limit max value can be changed on the command line</p> <div class="highlight-python"><pre>>ecflow_client --alter change limit_max <limit-name> <new-limit-value> <path-to-limit> >ecflow_client --alter change limit_max limit 2 /suite</pre> </div> <p>It can also be changed in python</p> <div class="highlight-python"><div class="highlight"><pre><span class="c">#!/usr/bin/env python2.7</span> <span class="kn">import</span> <span class="nn">ecflow</span> <span class="k">try</span><span class="p">:</span> <span class="n">ci</span> <span class="o">=</span> <span class="n">ecflow</span><span class="o">.</span><span class="n">Client</span><span class="p">()</span> <span class="n">ci</span><span class="o">.</span><span class="n">alter</span><span class="p">(</span><span class="s">"/suite"</span><span class="p">,</span><span class="s">"change"</span><span class="p">,</span><span class="s">"limit_max"</span><span class="p">,</span><span class="s">"limit"</span><span class="p">,</span> <span class="s">"2"</span><span class="p">)</span> <span class="k">except</span> <span class="ne">RuntimeError</span><span class="p">,</span> <span class="n">e</span><span class="p">:</span> <span class="k">print</span> <span class="s">"Failed: "</span> <span class="o">+</span> <span class="nb">str</span><span class="p">(</span><span class="n">e</span><span class="p">)</span> </pre></div> </div> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Limit" title="ecflow.Limit"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Limit</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_limit" title="ecflow.Node.add_limit"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_limit</span></tt></a>. For BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-limit"><tt class="xref std std-token docutils literal"><span class="pre">limit</span></tt></a> and <a class="reference internal" href="#term-inlimit"><em class="xref std std-term">inlimit</em></a></p> </dd> <dt id="term-manual-page">manual page</dt> <dd><p class="first">Manual pages are part of the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a>.</p> <p class="last">This is to ensure that the manual page is updated when the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> is updated. The manual page is a very important operational tool allowing you to view a description of a task, and possibly describing solutions to common problems. The <a class="reference internal" href="#term-pre-processing"><em class="xref std std-term">pre-processing</em></a> can be used to extract the manual page from the script file and is visible in <a class="reference internal" href="#term-ecflowview"><em class="xref std std-term">ecflowview</em></a>. The manual page is the text contained within the %manual and %end <a class="reference internal" href="#term-directives"><em class="xref std std-term">directives</em></a>. They can be seen using the manual button on <a class="reference internal" href="#term-ecflowview"><em class="xref std std-term">ecflowview</em></a>.</p> </dd> <dt id="term-meter">meter</dt> <dd><p class="first">The purpose of a meter is to signal proportional completion of a task and to be able to trigger another job which is waiting on this proportional completion.</p> <p>The meter is updated by placing the –meter <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a> in a <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a>.</p> <p>For python see: <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Meter" title="ecflow.Meter"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Meter</span></tt></a> and <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_meter" title="ecflow.Node.add_meter"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node.add_meter</span></tt></a>. For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-meter"><tt class="xref std std-token docutils literal"><span class="pre">meter</span></tt></a></p> <p class="last">Meter’s can be referenced in <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> and <a class="reference internal" href="#term-complete-expression"><em class="xref std std-term">complete expression</em></a> expressions.</p> </dd> <dt id="term-node">node</dt> <dd><p class="first"><a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a>, <a class="reference internal" href="#term-family"><em class="xref std std-term">family</em></a> and <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> form a hierarchy. Where a <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a> serves as the root of the hierarchy. The <a class="reference internal" href="#term-family"><em class="xref std std-term">family</em></a> provides the intermediate nodes, and the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> provide the leaf’s.</p> <p>Collectively <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a>, <a class="reference internal" href="#term-family"><em class="xref std std-term">family</em></a> and <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> can be referred to as nodes.</p> <p class="last">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node" title="ecflow.Node"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Node</span></tt></a>.</p> </dd> <dt id="term-pre-processing">pre-processing</dt> <dd><p class="first">Pre-processing takes place during <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> and acts on <a class="reference internal" href="#term-directives"><em class="xref std std-term">directives</em></a> specified in <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> file.</p> <p>This involves:</p> <blockquote class="last"> <div><p>o expanding any include file <a class="reference internal" href="#term-directives"><em class="xref std std-term">directives</em></a>. i.e similar to ‘c’ language pre-processing</p> <p>o removing comments and manual <a class="reference internal" href="#term-directives"><em class="xref std std-term">directives</em></a></p> <p>o performing <a class="reference internal" href="#term-variable-substitution"><em class="xref std std-term">variable substitution</em></a></p> </div></blockquote> </dd> <dt id="term-queued">queued</dt> <dd><p class="first">Is a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a>.</p> <p class="last">After the begin command, the task <strong>without</strong> a <a class="reference internal" href="#term-defstatus"><em class="xref std std-term">defstatus</em></a> are placed into the queued state</p> </dd> <dt id="term-real-clock">real clock</dt> <dd><p class="first">A <a class="reference internal" href="#term-suite"><em class="xref std std-term">suite</em></a> using a real <a class="reference internal" href="#term-clock"><em class="xref std std-term">clock</em></a> will have its <a class="reference internal" href="#term-clock"><em class="xref std std-term">clock</em></a> matching the clock of the machine. Hence the <a class="reference internal" href="#term-date"><em class="xref std std-term">date</em></a> advances by one day at midnight.</p> <p>If server is suspended, No jobs can be created. Issues:</p> <blockquote class="last"> <div><ul class="simple"> <li>relative time periods in time series can be missed</li> <li>Dependencies with time series, can result in missed time slots when the server is resumed any pending jobs are submitted straight away.</li> </ul> </div></blockquote> </dd> <dt id="term-repeat">repeat</dt> <dd><p class="first">Repeats provide looping functionality. There can only be a single repeat on a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>.</p> <blockquote> <div><p>repeat day step [ENDDATE] # only for suites</p> <p>repeat integer VARIABLE start end [step]</p> <p>repeat enumerated VARIABLE first [second [third ...]]</p> <p>repeat string VARIABLE str1 [str2 ...]</p> <p>repeat file VARIABLE filename</p> <p>repeat date VARIABLE yyyymmdd yyyymmdd [delta]</p> </div></blockquote> <p>The repeat VARIABLE can be used in <a class="reference internal" href="#term-trigger"><em class="xref std std-term">trigger</em></a> is evaluated. You are strongly advised to avoid cross-suite /em></a> and <a class="reference internal" href="#term-dependenciescomplete-expression"><em class="xref std std-term">dependencies<>complete expression</em></a> expressions.</p> <p class="last">Families>For and suites that depend on one another should be placed in a single python see <a class="reference internal" href="#term-suite"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_repeat" title="ecflow.Node.add_repeat"><tt class="xref stdpy std-term">suite</em></a>. If you think you need cross-suite dependencies, you should consider merging the suites together and have each as a top-level family in the merged suite. For BNF see py-class docutils literal"><span class="pre">ecflow.Node.add_repeat</span></tt></a>, <a class="reference internal" href="/wiki/display/ECFLOW/DefinitionecFlow+file+Grammer#grammar-token-externPython+Api#ecflow.Repeat" title="ecflow.Repeat"><tt class="xref stdpy stdpy-tokenclass docutils literal"><span class="pre">extern<>ecflow.Repeat</span></tt></a></p> </dd> <dt id="term-family">family</dt> <dd><pa>, <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.RepeatDate" title="ecflow.RepeatDate"><tt class="xref py py-class docutils literal"><span class="first">Is a pre">ecflow.RepeatDate</span></tt></a>, <a class="reference internal" href="#term-node"><em="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.RepeatEnumerated" title="ecflow.RepeatEnumerated"><tt class="xref py py-class docutils literal"><span class="xref std std-term">node</em></a> in a pre">ecflow.RepeatEnumerated</span></tt></a>, <a class="reference internal" href="#term-suite-definition"><em="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.RepeatInteger" title="ecflow.RepeatInteger"><tt class="xref stdpy std-term">suite definition</em></a>.</p> <p>A family is a collection of py-class docutils literal"><span class="pre">ecflow.RepeatInteger</span></tt></a>, <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>‘s and families.</p> <p class="last">Typically you place tasks that are related to each other inside the same family, analogous to the way you create directories to contain related files. For /wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.RepeatDay" title="ecflow.RepeatDay"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.RepeatDay</span></tt></a> For text BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammarGrammar#grammar-token-familyrepeat"><tt class="xref std std-token docutils literal"><span class="pre">family<>repeat</span></tt></a></p> </dd> <dt id="term-haltedrunning">halted<>running</dt> <dd><p class="first">Is<dd>Is a <a class="reference internal" href="#term-ecfecflow-server"><em class="xref std std-term">ecf>ecflow_server</em></a> state</p> <p>The following tables reflects the server capabilities in the different states</p> <table border="1" class="last docutils"> <colgroup> <col width="22%" /> <col width="16%" /> <col width="16%" /> <col width="19%" /> <col width="26%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">State</th> <th class="head">User Request</th> <th class="head">Task Request</th> <th class="head">Job Scheduling</th> <th class="head">Auto-Check-pointing</th> </tr> </thead> <tbody valign="top"> <tr><td><astate. See <a class="reference internal" href="#term-server-states"><em class="xref std std-term">server states</em></a></dd> <dt id="term-scheduling">scheduling</dt> <dd><p class="first">The <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> is responsible for <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> scheduling.</p> <p class="last">It will check <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a> in the <a class="reference internal" href="#term-suite-runningdefinition"><em class="xref std std-term">running<>suite definition</em></a></td> <td>yes</td> <td>yes</td> <td>yes</td> <td>yes</td> </tr> <tr><td><aa> every minute. If these <a class="reference internal" href="#term-shutdowndependencies"><em class="xref std std-term">shutdown<>dependencies</em></a></td> <td>yes</td> <td>yes</td> <td>no</td> <td>yes</td> </tr> <tr><td><aa> are free, the <a class="reference internal" href="#term-ecflow-haltedserver"><em class="xref std std-term">halted<>ecflow_server</em></a></td> <td>yes</td> <td>no</td> <td>no</td> <td>no</td> </tr> </tbody> </table> </dd> <dt id="term-inlimit">inlimit</dt> <dd><p class="first">The inlimit works in conjunction with /a> will submit the task. See <a class="reference internal" href="#term-job-limitcreation"><em class="xref std std-term">limit<>job creation</em></a> for providing simple load management</p> <p.</p> </dd> <dt id="term-server-states">server states</dt> <dd><p class="lastfirst">inlimit>The isfollowing addedtables toreflects the <a class="reference internal" href="#term-ecflow-nodeserver"><em class="xref std std-term">node<>ecflow_server</em></a> thatcapabilities needsin tothe bedifferent limited.<states</p> </dd> <dt id="term-job-creation">job creation</dt> <dd><p class="first">The process of job creation includes:</p> <blockquote> <div><p>o Locating <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> files , corresponding to the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> in the <a class="reference internal" href="#term-suite-definition"><em class="xref std std-term">suite definition</em></a></p> <p>o <a<table border="1" class="last docutils"> <colgroup> <col width="22%" /> <col width="16%" /> <col width="16%" /> <col width="19%" /> <col width="26%" /> </colgroup> <thead valign="bottom"> <tr class="row-odd"><th class="head">State</th> <th class="head">User Request</th> <th class="head">Task Request</th> <th class="head">Job Scheduling</th> <th class="head">Auto-Check-pointing</th> </tr> </thead> <tbody valign="top"> <tr class="row-even"><td><a class="reference internal" href="#term-pre-processingrunning"><em class="xref std std-term">pre-processing<>running</em></a></p>td> <td>yes</td> <td>yes</td> <td>yes</td> <td>yes</td> </div></blockquote>tr> <p>The steps above transforms an <a <tr class="row-odd"><td><a class="reference internal" href="#term-ecf-scriptshutdown"><em class="xref std std-term">ecf script<>shutdown</em></a> to a <a /a></td> <td>yes</td> <td>yes</td> <td>no</td> <td>yes</td> </tr> <tr class="row-even"><td><a class="reference internal" href="#term-job-filehalted"><em class="xref std std-term">job file<>halted</em></a> that can be submitted.</p> <p>The running jobs will communicate back to the /a></td> <td>yes</td> <td>no</td> <td>no</td> <td>no</td> </tr> </tbody> </table> </dd> <dt id="term-shutdown">shutdown</dt> <dd>Is a <a class="reference internal" href="#term-ecfecflow-server"><em class="xref std std-term">ecf>ecflow_server</em></a> bystate. callingSee <a class="reference internal" href="#term-childserver-commandstates"><em class="xref std std-term">child>server command<states</em></a>‘s.</p> <p class="last">This causes <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a> changes on the a></dd> <dt id="term-status">status</dt> <dd><p class="first">Each <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>‘s in the <a class="reference internal" href="#term-ecfsuite-serverdefinition"><em class="xref std std-term">ecf_server<>suite definition</em></a> andhas flags can be set to indicate various eventsa status.</p> </dd> <dt id="term-job-file">job file</dt> <dd><p class="first">The job file is created by<p>Status reflects the state of the <a class="reference internal" href="#term-ecf-servernode"><em class="xref std std-term">ecf_server<>node</em></a>. In during <a class="reference internal" href="#term-job-creationecflowview"><em class="xref std std-term">job creation<>ecflowview</em></a>.</p> <p>It is derived from the <a the background colour of the text reflects the status.</p> <p><a class="reference internal" href="#term-ecf-scripttask"><em class="xref std std-term">ecf script<>task</em></a> after expanding the pre-processingstatus are: <a class="reference internal" href="#term-directivesunknown"><em class="xref std std-term">directives<>unknown</em></a>.</p> <p, <a class="lastreference internal">It has the extension ”.job{try number}”, i.e. t1.job1</p> </dd> <dt id="term-label">label</dt> <dd>A label has a name and a value and is a way of displaying information in href="#term-queued"><em class="xref std std-term">queued</em></a>, <a class="reference internal" href="#term-ecflowviewsubmitted"><em class="xref std std-term">ecFlowview<>submitted</em></a> For BNF see, <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-label"><tt#term-active"><em class="xref std std-token docutils literal"><span term">active</em></a>, <a class="pre">label</span></tt></a></dd> <dt id="term-late">late</dt> <dd><preference internal" href="#term-complete"><em class="first">Definexref a tag for a node to be late.</p> <p class="last">Suites cannot be late, but you can define a late tag for submitted in a suite, to be inherited by the families and tasks. When a node is classified as being late, the only action <a std std-term">complete</em></a>, <a class="reference internal" href="#term-aborted"><em class="xref std std-term">aborted</em></a> and <a class="reference internal" href="#term-suspended"><em class="xref std std-term">suspended</em></a></p> <p class="last"><a class="reference internal" href="#term-ecfecflow-server"><em class="xref std std-term">ecf>ecflow_server</em></a> takesstatus is to set a flag. are: <a class="reference internal" href="#term-ecflowviewshutdown"><em class="xref std std-term">ecFlowview<>shutdown</em></a> will display these alongside the, <a class="reference internal" href="#term-halted"><em class="xref std std-term">halted</em></a>, <a class="reference internal" href="#term-noderunning"><em class="xref std std-term">node<>running</em></a> name as an icon (and optionally pop up a window). For BNF see this is shown on the root node in <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-late"><tt#term-ecflowview"><em class="xref std std-token docutils literal"><span class="pre">late</span></tt></term">ecflowview</em></a></p> </dd> <dt id="term-limitsubmitted">limit<>submitted</dt> <dd>limit provides<dd><p class="first">Is a means of providing simple load management by say limiting the number of tasks submitted to a specific server. Typically you either define limits on <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> <a class="reference internal" href="#term-status"><em class="xref std std-term">status</em></a>.</p> <p class="last">When the <a class="reference internal" href="#term-suitetask"><em class="xref std std-term">suite<>task</em></a> level or define a separate suite to hold limits so that they can be used by multiple suites. For BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-limit"><tt#term-dependencies"><em class="xref std std-token docutils literal"><span class="pre">limit</span></tt></a> andterm">dependencies</em></a> are resolved/free the <a class="reference internal" href="#term-inlimitecflow-server"><em class="xref std std-term">inlimit<>ecflow_server</em></a></dd> <dt id="term-manual-page">manual page</dt> <dd><p class="first">Manual pages are part ofa> places the task into a submitted state. However if the ECF_JOB_CMD fails, the task is placed into the <a class="reference internal" href="#term-ecf-scriptaborted"><em class="xref std std-term">ecf script<>aborted</em></a>.</p> <p state</p> </dd> <dt id="term-suite">suite</dt> <dd><p class="lastfirst">This>A is to ensure that the manual page suite is updated when the script is updated. The manual pageorganisational entity. It is aserves veryas importantthe operational tool allowing you to view a description of a task, and possibly describing solutions to common problems. The root <a class="reference internal" href="#term-pre-processingnode"><em class="xref std std-term">pre-processing</em></a> can be used to extract the manual page from the script file and is visible in >node</em></a> in a <a class="reference internal" href="#term-ecflowviewsuite-definition"><em class="xref std std-term">ecFlowview<>suite definition</em></a>. The manual page is the text contained within the %manual and %endIt should be used to hold a set of jobs that achieve a common function. It can be used to hold user <a class="reference internal" href="#term-directivesvariable"><em class="xref std std-term">directives<>variable</em></a> s that are common to all of its children.</p> They<p>Only cana besuite seennode usingcan thehave manuala button on <a class="reference internal" href="#term-ecflowviewclock"><em class="xref std std-term">ecFlowview<>clock</em></a>.</p> </dd> <dt id<p class="term-meter">meter</dt> <dd>The purpose of a meter is to signal proportional completion of a task and to be able to trigger another job which is waiting on this proportional completion For BNF seelast">It is a collection of <a class="reference internal" href="#term-family"><em class="xref std std-term">family</em></a>‘s, <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-meter"><tt#term-variable"><em class="xref std std-token docutils literal"><span class="pre">meter</span></tt></a></dd> <dt id="term-node">node</dt> <dd>A node is a term">variable</em></a>‘s, <a class="reference internal" href="#term-repeat"><em class="xref std std-term">repeat</em></a> and a single <a class="reference internal" href="#term-suiteclock"><em class="xref std std-term">suite</em></a>,>clock</em></a> definition. For a complete list of attributes look at BNF for <a class="reference internal" href="#term-family"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-suite"><tt class="xref std std-term">family</em></a> ortoken docutils literal"><span class="pre">suite</span></tt></a>. For python see <a class="reference internal" href="#term-task"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Suite" title="ecflow.Suite"><tt class="xref stdpy std-term">task</em></a><py-class docutils literal"><span class="pre">ecflow.Suite</span></tt></a>.</p> </dd> <dt id="term-presuite-processing">pre-processing<definition">suite definition</dt> <dd><p class="first">Pre-processing takes place during <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> and acts on>The suite definition is the hierarchical <a class="reference internal" href="#term-directivesnode"><em class="xref std std-term">directives<>node</em></a> specified intree.</p> <p>It describes how your <a class="reference internal" href="#term-ecf-scripttask"><em class="xref std std-term">ecf script<>task</em></a> file‘s run and interact.</p> <p>This involves<p>It can built up using:</p> <blockquote<blockquote> <div><ul> <li><p class="last"> <div><p>o expanding any includes file directives. i.e similar to ‘c’ language pre-processing</p> <p>o removing comments and manual directives</p> <p>o performing <afirst">Ascii text file by following the rules defined in the ecFlow <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammar#grammer"><em>Definition file Grammar</em></a>.</p> <p>Hence any language can be used, to generate this format.</p> </li> <li><p class="first"><a class="reference internal" href="#term-variable-substitution"><em class="xref std std-term">variable substitution</wiki/display/ECFLOW/ecFlow+Python+Api#suite-definition-python-api"><em>suite definition python API</em></a></p> </div><li> </blockquote>ul> </dd> <dt id="term-queued">queued</dt> <dd><pdiv></blockquote> <p class="first">Is a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>last">Once the definition is built, it can be loaded into the <a class="reference internal" href="#term-statusecflow-server"><em class="xref std std-term">status<>ecflow_server</em></a>.</p> <p class="last">After the begin command, the task <strong>without</strong> a, and started. It can be monitored by <a class="reference internal" href="#term-defstatusecflowview"><em class="xref std std-term">defstatus<>ecflowview</em></a> are placed into the queued state<a></p> </dd> <dt id="term-repeatsuspended">repeat<>suspended</dt> <dd><p class="first">Repeats provide looping functionality. There can only be a single repeat on a >Is a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a>.</p> <blockquote> <div><p>repeat day step [ENDDATE] # only for suites</p> <p>repeat integer VARIABLE start end [step]</p> <p>repeat enumerated VARIABLE first [second [third ...]]</p> <p>repeat string VARIABLE str1 [str2 ...]</p> <p>repeat file VARIABLE filename</p> <p>repeat date VARIABLE yyyymmdd yyyymmdd [delta]</p> </div></blockquote> <p class="last">The repeat VARIABLE can be used in state. A <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> can be placed into the suspended state via a <a class="reference internal" href="#term-defstatus"><em class="xref std std-term">defstatus</em></a> or via <a class="reference internal" href="#term-triggerecflowview"><em class="xref std std-term">trigger<>ecflowview</em></a> anda></p> <p class="last">A suspended <a class="reference internal" href="#term-complete-triggernode"><em class="xref std std-term">complete trigger<>node</em></a> expressions For BNF see including any of its children can not take part in <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-repeat"><tt#term-scheduling"><em class="xref std std-token docutils literal"><span class="pre">repeat</span></tt></a></term">scheduling</em></a> until the node is resumed.</p> </dd> <dt id="term-runningtask">running<>task</dt> <dd><p class="first">Is a>A task represents a job that needs to be carried out. It serves as a leaf <a class="reference internal" href="#term-ecf-servernode"><em class="xref std std-term">ecf_server<>node</em></a> state.</p> <p>The following tables reflects the server capabilities in thea different states</p> <table border="1" <a class="lastreference docutilsinternal"> <colgroup> <col widthhref="22%#term-suite-definition" /> <col width><em class="16%" /> <col width="16%" /> <col width="19%" /> <col width="26%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">State</th> <th class="head">User Request</th> <th class="head">Task Request</th> <th class="head">Job Scheduling</th> <th class="head">Auto-Check-pointing</th> </tr> </thead> <tbody valign="top"> <tr><td><axref std std-term">suite definition</em></a></p> <p>Only tasks can be submitted.</p> <p>A job inside a task <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> should generally be re-entrant so that no harm is done by rerunning it, since a task may be automatically submitted more than once if it aborts.</p> <p class="last">For python see <a class="reference internal" href="#term-running"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Task" title="ecflow.Task"><tt class="xref stdpy std-term">running</em></a></td> <td>yes</td> <td>yes</td> <td>yes</td> <td>yes</td> </tr> <tr><td><apy-class docutils literal"><span class="pre">ecflow.Task</span></tt></a>. For text BNF see <a class="reference internal" href="#term-shutdown"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-task"><tt class="xref std std-term">shutdown</em></a></td> <td>yes</td> <td>yes</td> <td>no</td> <td>yes</td> </tr> <tr><td><a class="reference internal" href="#term-halted"><emtoken docutils literal"><span class="xref std std-term">halted</em><pre">task</span></tt></a></td>p> <td>yes<</td> <td>no</td> <td>no</td> <td>no</td> </tr> </tbody> </table> </dd> <dt id="term-scheduling">scheduling</dt> <dd><p class="first">The <a class="reference internal" href="#term-ecf-server"><em class="xref std std-term">ecf_server</em></a> is responsible for <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a> scheduling.</p> <p class="last">It will check <a class="reference internal" href="#term-dependencies"><em class="xref std std-term">dependencies</em></a> in thedd> <dt id="term-time">time</dt> <dd><p class="first">This defines a time dependency for a node.</p> <p>Time is expressed in the format [h]h:mm. Only numeric values are allowed. There can be multiple time dependencies for a node, but overlapping times may cause unexpected results. To define a series of times, specify the start time, end time and a time increment. If the start time begins with ‘+’, times are relative to the beginning of the suite or, in repeated families, relative to the beginning of the repeated family.</p> <p class="last">For python see <a class="reference internal" href="#term-suite-definition"><em="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Time" title="ecflow.Time"><tt class="xref stdpy std-term">suite definition</em><py-class docutils literal"><span class="pre">ecflow.Time</span></tt></a> every minute. If these and <a class="reference internal" href="#term-dependencies"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_time" title="ecflow.Node.add_time"><tt class="xref stdpy std-term">dependencies</em></a> are free, the <a class="reference internal" href="#term-ecf-server"><em class="xref std std-term">ecf_server</em></a> will submit the task. Seepy-class docutils literal"><span class="pre">ecflow.Node.add_time</span></tt></a>. For BNF see <a class="reference internal" href="#term-job-creation"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-time"><tt class="xref std std-term">job creation</em></a>.<token docutils literal"><span class="pre">time</span></tt></a></p> </dd> <dt id="term-shutdowntoday">shutdown<>today</dt> <dd><p class="first">Is a>Like <a class="reference internal" href="#term-ecf-servertime"><em class="xref std std-term">ecf_server<>time</em></a> state, but “today” does not wrap to tomorrow.</p> <p>The following tables reflects the server capabilities in the different states</p> <table border="1"<p>If suites’ begin time is past the time given for the “today” command the node is free to run (as far as the time dependency is concern.)</p> <p class="last docutils"> <colgroup> <col width="22%" /> <col width="16%" /> <col width="16%" /> <col width="19%" /> <col width="26%" /> </colgroup> <thead valign="bottom"> <tr><th class="head">State</th> <th class="head">User Request</th> <th class="head">Task Request</th> <th class="head">Job Scheduling</th> <th class="head">Auto-Check-pointing</th> </tr> </thead> <tbody valign="top"> <tr><td><a">For python see <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Today" title="ecflow.Today"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.Today</span></tt></a>. For text BNF see <a class="reference internal" href="#term-running"><em/wiki/display/ECFLOW/Definition+file+Grammar#grammar-token-today"><tt class="xref std std-term">running</em></a></td> <td>yes</td> <td>yes</td> <td>yes</td> <td>yes</td> </tr> <tr><td><a class="reference internal" href="#term-shutdown"><em class="xref std std-term">shutdown</em></a></td> <td>yes</td> <td>yes</td> <td>no</td> <td>yes</td> </tr> <tr><td><atoken docutils literal"><span class="pre">today</span></tt></a></p> </dd> <dt id="term-trigger">trigger</dt> <dd><p class="first">Triggers defines a dependency for a task or family.</p> <p>There can be only one trigger dependency per node, but that can be a complex boolean expression of the <a class="reference internal" href="#term-haltedstatus"><em class="xref std std-term">halted<>status</em></a></td> <td>yes</td> <td>no</td> <td>no</td> <td>no</td> </tr> </tbody> </table> </dd> <dt id="term-status">status</dt> <dd><p class="first">Eacha> of several nodes. Triggers should be avoided on suites. A node with a trigger can only be activated when its trigger has expired. A trigger holds the node as long as the trigger’s expression evaluation returns false.</p> <p>The keywords in trigger expressions are: <a class="reference internal" href="#term-nodeunknown"><em class="xref std std-term">node<>unknown</em></a> in, <a class="reference internal" href="#term-suite-definitionsuspended"><em class="xref std std-term">suite definition<>suspended</em></a> has a status.</p> <p>Status reflects the state of the , <a class="reference internal" href="#term-nodecomplete"><em class="xref std std-term">node<>complete</em></a>. In, <a class="reference internal" href="#term-ecflowviewqueued"><em class="xref std std-term">ecFlowview<>queued</em></a>, the background colour of the text reflects the status.</p> <p><a <a class="reference internal" href="#term-tasksubmitted"><em class="xref std std-term">task<>submitted</em></a>, status are: <a class="reference internal" href="#term-unknownactive"><em class="xref std std-term">unknown<>active</em></a>, <a class="reference internal" href="#term-queuedaborted"><em class="xref std std-term">queued<>aborted</em></a>, <a class="reference internal" href="#term-submitted"><em and <strong>clear</strong> and <strong>set</strong> for event status.</p> <p class="xref std std-term">submitted</em></a>,last">For python see <a class="reference internal" href="#term-active"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Expression" title="ecflow.Expression"><tt class="xref stdpy std-term">active</em></a>, <a class="reference internal" href="#term-complete"><empy-class docutils literal"><span class="xref std std-term">complete</em></a>,pre">ecflow.Expression</span></tt></a> and <a class="reference internal" href="#term-aborted"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_trigger" title="ecflow.Node.add_trigger"><tt class="xref stdpy std-term">aborted</em></a> and <apy-class docutils literal"><span class="reference internal" href="#term-suspended"><em class="xref std std-term">suspended</em></a></p> <ppre">ecflow.Node.add_trigger</span></tt></a></p> </dd> <dt id="term-unknown">unknown</dt> <dd><p class="last"><afirst">Is a <a class="reference internal" href="#term-ecf-servernode"><em class="xref std std-term">ecf_server<>node</em></a> status are: <a class="reference internal" href="#term-shutdownstatus"><em class="xref std std-term">shutdown<>status</em></a>, <a .</p> <p class="reference internallast" href="#term-halted"><em class="xref std std-term">halted</em></a>, >This is the default <a class="reference internal" href="#term-runningnode"><em class="xref std std-term">running<>node</em></a> this is shown on the root node in <a class="reference internal" href="#term-ecflowviewstatus"><em class="xref std std-term">ecFlowview<>status</em></a></p> </dd> <dt id="term-submitted">submitted</dt> <dd><p class="first">Isa> when a <a class="reference internal" href="#term-suite-nodedefinition"><em class="xref std std-term">node<>suite definition</em></a>a> is loaded into the <a class="reference internal" href="#term-ecflow-statusserver"><em class="xref std std-term">status<>ecflow_server</em></a>.</p> <pa></p> </dd> <dt id="term-variable">variable</dt> <dd><p class="last">When the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>first">ECF makes heavy use of different kinds of variables.There are several kinds of variables:</p> <blockquote> <div><p>Environment variables: which are set in the UNIX shell before the <a class="reference internal" href="#term-dependenciesecflow"><em class="xref std std-term">dependencies<>ecFlow</em></a> are resolved/free thestarts. These control <a class="reference internal" href="#term-ecfecflow-server"><em class="xref std std-term">ecf>ecflow_server</em></a> places the task into a submitted state. However if the ECF_JOB_CMD fails, the task is placed into the /em></a>, and <a class="reference internal" href="#term-ecflow-abortedclient"><em class="xref std std-term">aborted<>ecflow_client</em></a> state< .</p> </dd> <dt id="term-suite">suite</dt> <dd><p class="first">Is a<p>suite definition variables: Also referred to as user variables. These control <a class="reference internal" href="#term-ecflow-nodeserver"><em class="xref std std-term">node<>ecflow_server</em></a>, in aand <a class="reference internal" href="#term-suiteecflow-definitionclient"><em class="xref std std-term">suite definition<>ecflow_client</em></a></p> <p class="last">A suite is a collection ofa> and are available for use in <a class="reference internal" href="#term-familyjob-file"><em class="xref std std-term">family</em></a>‘s, std std-term">job file</em></a>.</p> <p>Generated variables: These are generated within the suite definition node tree during <a class="reference internal" href="#term-variablejob-creation"><em class="xref std std-term">variable<>job creation</em></a>‘s, and are available for use in the <a class="reference internal" href="#term-job-repeatfile"><em class="xref std std-term">repeat<>job file</em></a> and .</p> </div></blockquote> <p class="last">For python see <a class="reference internal" href="#term-clock"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.Node.add_variable" title="ecflow.Node.add_variable"><tt class="xref stdpy std-term">clock</em></a> definitions. For a complete list of attributes look at BNF forpy-class docutils literal"><span class="pre">ecflow.Node.add_variable</span></tt></a>. For BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammarGrammar#grammar-token-suitevariable"><tt class="xref std std-token docutils literal"><span class="pre">suite<>variable</span></tt></a></p> </dd> <dt id="term-suitevariable-definitioninheritance">suite>variable definition<inheritance</dt> <dd><p class="first">The suite definition>When a <a class="reference internal" href="#term-variable"><em class="xref std std-term">variable</em></a> is theneeded hierarchicalat <a class="reference internal" href="#term-nodejob-creation"><em class="xref std std-term">node<>job creation</em></a> tree.</p> <p>It describes how yourtime, it is first sought in the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>‘s run and interact itself.</p> <p>It can built up using</p> <blockquote> <div><ul> <li><p class="first">Ascii text file by following the rules defined in the ecFlow<p>If it is not found in the <a class="reference internal" href="#term-task"><em class="xref std std-term">task</em></a>, it is sought from the task’s parent and so on, up through the <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammer"><em>Definition file Grammer<#term-node"><em class="xref std std-term">node</em></a>.</p> <p>Hence any language can be used, to generate this format levels until found.</p> </li> <li><p class="first"><a<p>For any <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#python-api"><em>ecFlow Python Api</em></a></p> </li> </ul> </div></blockquote>#term-node"><em class="xref std std-term">node</em></a>, there are two places to look for variables.</p> <p class="last">Once>Suite the definition isvariables built,are itlooked isfor loadedfirst, intoand thethen <a class="reference internal" href="#term-ecf-server"><em class="xref std std-term">ecf_server</em></a>, and started. It is then monitored byany generated variables.</p> </dd> <dt id="term-variable-substitution">variable substitution</dt> <dd><p class="first">Takes place during <a class="reference internal" href="#term-ecflowviewpre-processing"><em class="xref std std-term">ecFlowview<>pre-processing</em></a></p> </dd> <dt id="term-suspended">suspended</dt> <dd><p class="first">Is aa> <em>or</em> command invocation.(i.e ECF_JOB_CMD,ECF_KILL_CMD,etc)</p> <p>It involves searching each line of <a class="reference internal" href="#term-ecf-nodescript"><em class="xref std std-term">node<>ecf script</em></a> state. A file or command, for <a class="reference internal" href="#term-ecf-nodemicro"><em class="xref std std-term">node<>ECF_MICRO</em></a> can be placed into the suspended state via a character. typically ‘%’</p> <p>The text between two % character, defines a variable. i.e %VAR%</p> <p>This variable is searched for in the <a class="reference internal" href="#term-suite-defstatusdefinition"><em class="xref std std-term">defstatus</em></a> or via>suite definition</em></a>.</p> <p>First the suite definition variables( sometimes referred to as user variables) are searched and then Repeat variable name, and finally the generated variables.If no variable is found then the same search pattern is repeated up the node tree.</p> <p>The value of the <a class="reference internal" href="#term-ecflowviewvariable"><em class="xref std std-term">ecFlowview<>variable</em></a><a> is replaced between the % characters.</p> <p class="last">A suspended<p>If the variable is not found in the <a class="reference internal" href="#term-suite-nodedefinition"><em class="xref std std-term">node<>suite definition</em></a> including any of its children can not take part in during pre-processing then <a class="reference internal" href="#term-schedulingjob-creation"><em class="xref std std-term">scheduling<>job creation</em></a> until the node is resumed.</p> </dd> <dt id="term-task">task</dt> <dd><p class="first">Is a <a class="reference internal" href="#term-node"><em class="xref std std-term">node</em></a> in afails, and an error message is written to the log file, and the task is placed into the <a class="reference internal" href="#term-suite-definitionaborted"><em class="xref std std-term">suite definition<>aborted</em></a>, that represents a job that needs to be carried out state.</p> <p>Only<p>To tasksavoid canthis, be submitted.</p> <p class="last">A job inside a task variables in the <a class="reference internal" href="#term-ecf-script"><em class="xref std std-term">ecf script</em></a> shouldcan generallybe be re-entrant so that no harm is done by rerunning it, since a task may be automatically submitted more than once if it aborts For BNF seedefined as:</p> <blockquote class="last"> <div>%VAR:replacement% : This is similar to %VAR% but if VAR is not found in the <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-task"><tt"#term-suite-definition"><em class="xref std std-token docutils literal"><span class="pre">task</span></tt></a></p>term">suite definition</em></a> then ‘replacement’ is used.</div></blockquote> </dd> <dt id="term-time">time<virtual-clock">virtual clock</dt> <dd><p class="first">This>Like defines a time dependency for a node.</p> <p class="last">Time is expressed in the format [h]h:mm. Only numeric values are allowed. There can be multiple time dependencies for a node, but overlapping times may cause unexpected results. To define a series of times, specify the start time, end time and a time increment. If the start time begins with ‘+’, times are relative to the beginning of the suite or, in repeated families, relative to the beginning of the repeated family. For BNF see <a class="reference internal" href="#term-real-clock"><em class="xref std std-term">real clock</em></a> until the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> is suspended (i.e <a class="reference internal" href="#term-shutdown"><em class="xref std std-term">shutdown</em></a> or <a class="reference internal" href="#term-halted"><em class="xref std std-term">halted</em></a>), the suites <a class="reference internal" href="#term-clock"><em class="xref std std-term">clock</em></a> is also suspended.</p> <p>Hence will honour relative times in <a class="reference internal" href="#term-cron"><em class="xref std std-term">cron</em></a>, <a class="reference internal" href="#term-today"><em class="xref std std-term">today</em></a> and <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-time"><tt class="xref std std-token docutils literal"><span class="pre">time</span></tt></a></#term-time"><em class="xref std std-term">time</em></a> dependencies. It is possible to have a combination of hybrid/real and virtual.</p> <p class="last">More useful when we want complete adherence to time related dependencies at the expense being out of sync with system time.</p> </dd> <dt id="term-todayzombie">today<>zombie</dt> <dd><p class="first">Like>Zombies are running jobs that fail authentication when communicating with the <a class="reference internal" href="#term-ecflow-timeserver"><em class="xref std std-term">time<>ecflow_server</em></a>, but “today” does not wrap to tomorrow.<a></p> <p<p><a class="lastreference internal">If suites’ begin time is past the time given for the “today” command the node is free to run (as far as the time dependency is concern.) For BNF see href="#term-child-command"><em class="xref std std-term">child command</em></a> s like (init, event,meter, label, abort,complete) are placed in the <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-today"><tt#term-ecf-script"><em class="xref std std-token docutils literal"><spanterm">ecf script</em></a> file and are used to communicate with the <a class="pre">today</span></tt></a></p> </dd> <dt id="term-trigger">trigger</dt> <dd><preference internal" href="#term-ecflow-server"><em class="first">Triggersxref defines a dependency for a task or familystd std-term">ecflow_server</em></a>.</p> <p<p>The <a class="lastreference internal">There can be only one trigger dependency per node, but that can be a complex boolean expression of the status of several nodes. Triggers should be avoided on suites. A node with a trigger can only be activated when its trigger has expired. A trigger holds the node as long as the trigger’s expression evaluation returns false.</p> </dd> <dt id="term-unknown">unknown</dt> <dd><p class="first">Is a href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a> authenticates each connection attempt made by the <a class="reference internal" href="#term-child-command"><em class="xref std std-term">child command</em></a>. Authentication can fail for a number of reasons:</p> <blockquote> <div><ul class="simple"> <li>password(ECF_PASS) supplied with the child command, does not match the one in the <a class="reference internal" href="#term-ecflow-server"><em class="xref std std-term">ecflow_server</em></a></li> <li>path name(ECF_NAME) supplied with the child command, does not locate a task in the <a class="reference internal" href="#term-ecflow-nodeserver"><em class="xref std std-term">node</em></a>"xref std std-term">ecflow_server</em></a></li> <li>process id(ECF_RID) supplied with child command, does not correspond with the one stored in the <a class="reference internal" href="#term-ecflow-statusserver"><em class="xref std std-term">status<>ecflow_server</em></a>.<a></p>li> <p class="last">This<li>if a task is the defaultalready <a class="reference internal" href="#term-nodeactive"><em class="xref std std-term">node<>active</em></a>, but receives another init <a class="reference internal" href="#term-child-statuscommand"><em class="xref std std-term">status<>child command</em></a> when a /a></li> </ul> </div></blockquote> <p>When authentication fails the job is considered to be a zombie. The zombie is held in the <a class="reference internal" href="#term-suiteecflow-definitionserver"><em class="xref std std-term">suite definition</em></a> is loaded into the >ecflow_server</em></a> for a period of time, before it is automatically removed However the removed zombie, may well re-appear.</p> <p>For python see <a class="reference internal" href="#term-ecf-server"><em class="xref std std-term">ecf_server</em></a></p> </dd> <dt id="term-variable">variable</dt> <dd><p class="first">ECF makes heavy use of different kinds of variables.There are several kinds of variables:</p> <blockquote> <div><p>Environment variables: which are set in the UNIX shell before the <a class="reference internal" href="#term-ecflow"><em/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.ZombieAttr" title="ecflow.ZombieAttr"><tt class="xref py py-class docutils literal"><span class="pre">ecflow.ZombieAttr</span></tt></a>, <a class="reference internal" href="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.ZombieUserActionType" title="ecflow.ZombieUserActionType"><tt class="xref stdpy std-term">ecFlow</em></a> starts. These control <apy-class docutils literal"><span class="reference internal" href="#term-ecf-server"><em class="xref std std-term">ecf_server</em></a>, andpre">ecflow.ZombieUserActionType</span></tt></a></p> <p class="last">There are several types of zombies see <a class="reference internal" href="#term-ecfzombie-clienttype"><em class="xref std std-term">ecf_client<>zombie type</em></a> .</p> <p>suite definition variables: Also referred to as user variables. These control and <a class="reference internal" href="#term-ecf-server"><em="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.ZombieType" title="ecflow.ZombieType"><tt class="xref stdpy std-term">ecf_server</em></a>, and <apy-class docutils literal"><span class="reference internal" href="#term-ecf-client"><em class="xref std std-term">ecf_client</em></a> and are available for use inpre">ecflow.ZombieType</span></tt></a></p> </dd> <dt id="term-zombie-attribute">zombie attribute</dt> <dd><p class="first">The zombie attribute defines how a <a class="reference internal" href="#term-job-filezombie"><em class="xref std std-term">job file<>zombie</em></a>.</p> <p>Generated variables: These are generated within the suite definition node tree during <a class="reference internal" href="#term-job-creation"><em class="xref std std-term">job creation</em></a> and are available for use in the should be handled in an automated fashion. Very careful consideration should be taken before this attribute is added as it may hide a genuine problem. It can be added to any <a class="reference internal" href="#term-job-filenode"><em class="xref std std-term">job file<>node</em></a>.</p> </div></blockquote> <p class="last">For BNF see <a class="reference internal" href="/wiki/display/ECFLOW/Definition+file+Grammer#grammar-token-variable"><tt class="xref std std-token docutils literal"><span class="pre">variable</span></tt></a></p> </dd> <dt id="term-variable-inheritance">variable inheritance</dt> <dd><p class="first">When a But is best defined at the <a class="reference internal" href="#term-variablesuite"><em class="xref std std-term">variable<>suite</em></a> is needed ator <a class="reference internal" href="#term-job-creationfamily"><em class="xref std std-term">job creation<>family</em></a> time, itlevel. If there is first sought in no zombie attribute the default behaviour is to block the <a class="reference internal" href="#term-child-taskcommand"><em class="xref std std-term">task<>child command</em></a> itself.</p> <p>If it is not found in the<p class="last">To add a zombie attribute in python, please see: <a class="reference internal" href="#term-task"><em"/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.ZombieAttr" title="ecflow.ZombieAttr"><tt class="xref stdpy std-term">task</em></a>, it is sought from the task’s parent and so on, up through thepy-class docutils literal"><span class="pre">ecflow.ZombieAttr</span></tt></a></p> </dd> <dt id="term-zombie-type">zombie type</dt> <dd><p class="first">See <a class="reference internal" href="#term-nodezombie"><em class="xref std std-term">node<>zombie</em></a> levels until found.</p> <p>For anyand class <a class="reference internal" href="#term-node"><em="/wiki/display/ECFLOW/ecFlow+Python+Api#ecflow.ZombieAttr" title="ecflow.ZombieAttr"><tt class="xref stdpy std-term">node</em></a>, there are two places to look for variables.</p> <p class="last">Suite definition variables are looked for first, and then any generated variables.</p> </dd> <dt id="term-variable-substitution">variable substitution</dt> <dd><p class="first">Takes place duringpy-class docutils literal"><span class="pre">ecflow.ZombieAttr</span></tt></a> for further information. There are several types of zombies</p> <blockquote class="last"> <div><ul> <li><dl class="first docutils"> <dt>path</dt> <dd><ul class="first last simple"> <li>The task path can not be found in the server, because node tree was deleted, replaced,reload, server crashed or backup server does not have node tree.</li> <li>Jobs could have been created, via server <a class="reference internal" href="#term-pre-processingscheduling"><em class="xref std std-term">pre-processing<>scheduling</em></a></p> <p>It involves searching each line of <aa> or ‘user’ commands</li> </ul> </dd> </dl> </li> <li><dl class="referencefirst internal" href="#term-ecf-script"><emdocutils"> <dt>user</dt> <dd><p class="xref std std-term">ecf script</em></a> file, for ECF_MICRO character. typically ‘%’</p> <p>The text between two % character, defines a variable. i.e %VAR%</p> <p>This variable is searched for in the"first last">job is created by user action like, rerun, re-queue. User zombies are differentiated from server(scheduled) since they are automatically created when the force option is used and we have tasks in an <a class="reference internal" href="#term-suite-definitionactive"><em class="xref std std-term">suite definition</em></a>.</p> <p>First the suite definition variables( sometimes referred to as user variables) are searched and then the generated variables.</p> <p>The value of the >active</em></a> or <a class="reference internal" href="#term-variablesubmitted"><em class="xref std std-term">variable<>submitted</em></a> is replaced between the % charactersstates.</p> <p>If the variable is not found in the </dd> </dl> </li> <li><dl class="first docutils"> <dt>ecf</dt> <dd><p class="first">Jobs are created as part of the normal <a class="reference internal" href="#term-suite-definitionscheduling"><em class="xref std std-term">suite definition<>scheduling</em></a> during pre-processing thena></p> <ul class="last simple"> <li>Server crashed ( or terminated and restarted) and the recovered <a class="reference internal" href="#term-jobcheck-creationpoint"><em class="xref std std-term">job>check creation<point</em></a> fails, and an error message file is writtenout to the log file, and the task is placed into theof date.</li> <li>A <a class="reference internal" href="#term-abortedtask"><em class="xref std std-term">aborted<>task</em></a> stateis repeatedly re-run, earlier copies will not be remembered.</p>li> <p>To<li>Job avoidsent this variables in theby another <a class="reference internal" href="#term-ecfecflow-scriptserver"><em class="xref std std-term">ecf script<>ecflow_server</em></a>, canbut bewhich defined as:</p> <p class="last">%VAR:replacement% : This is similar to %VAR% but if VAR is not found in the can not talk to the original <a class="reference internal" href="#term-suiteecflow-definitionserver"><em class="xref std std-term">suite definition<>ecflow_server</em></a></a> then ‘replacement’ is used.</p>li> <li>Network glitches/network down</li> </ul> </dd> </dl> </li> </ul> </div></blockquote> </dd> </dl> </div> |
Page History
Overview
Community Forums
Content Tools