Page History

<div class="section" id="task-wrappers-migration">
<span id="task-wrappers"></span><span id="index-0"></span>
<p>The task wrapper file does not normally need many changes,
if the task designer sticks to the KISS principle, focusing on
the functional aspect of the task.</p>
<ul>
<li><p class="first">The file name is changed, ending with <strong>.ecf</strong> instead of <strong>.sms</strong>.</p>
</li>
<li><p class="first">simply copy or link the original file from .sms into .ecf</p>
</li>
<li><p class="first">alternatively, define a variable ECF_EXTN in the definition
file:: edit ECF_EXTN .sms</p>
<p>This requests that the ecFlow server uses .sms wrappers as the task template.
In some cases, no files will need translation (no SMS variables, no
CDP calls)</p>
</li>
<li><p class="first">smsmicro is replaced with <strong>ecf_micro</strong>, when needed</p>
<table border="1" class="docutils">
<colgroup>
<col width="26%" />
<col width="29%" />
<col width="44%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head"><p class="first last">SMS</p>
</th>
<th class="head"><p class="first last">ecFlow</p>
</th>
<th class="head"><p class="first last">location</p>
</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td><p class="first last">SMSMICRO</p>
</td>
<td><p class="first last">ECF_MICRO</p>
</td>
<td><p class="first last">definition file</p>
</td>
</tr>
<tr class="row-odd"><td><p class="first last">%smsmicro</p>
</td>
<td><p class="first last">%ecf_micro</p>
</td>
<td><p class="first last">script .ecf .h</p>
</td>
</tr>
</tbody>
</table>
</li>
</ul>
<ul>
<li><p class="first">In ECMWF Operations, in the main branch, amongst 1394 files, only 43 use SMS
system variables, i.e. variables whose name starts with SMS. Among all the
suites MetApps is in charge of, amongst 3738 files, 216 are affected.
Extracting these variables, we have:</p>
<div class="highlight-python"><pre>============
%SMS in .sms
============
SMSCHECK
SMSCHECKOLD
SMSDATE
SMSFILES
SMSHOME
SMSHOST
SMSINCLUDE
SMSJOBOUT
SMSLOG
SMSNAME
SMSNODE
SMSTRYNO
SMSURLBASE
SMS_PROG
============</pre>
</div>
<dl class="docutils">
<dt>You must then decide whether to:</dt>
<dd><ul class="first last simple">
<li>tranlate SMS variables to the matching ecFlow variables</li>
<li>or replace SMS variable with a shell variable $SMS, or $ECF_</li>
</ul>
</dd>
</dl>
<p>Similarly, we can identify all scripts that call the CDP text client.</p>
<p>It is a good design principle to create tasks that are independent of SMS
system variables. Only the tasks in charge of &#8220;advanced use&#8221; are
concerned: SMSTRYNO was used to make a job aware of its instance
number, enabling verbose output in case of rerun.</p>
<p>One step translation consists of running the scripts through a
filter that can be used for both expanded SMS definition files or for
task wrappers:</p>
<div class="highlight-python"><pre>&gt; sed -f sms2ecf-min.sed X.sms &gt; X.ecf</pre>
</div>
<div class="highlight-python"><pre>#!/bin/sed -f 
/^  *action  */d 
/^  *edit ECF_DATE  */d
s:SMSNAME:ECF_NAME:g
s:SMSNODE:ECF_NODE:g
s:SMSPASS:ECF_PASS:g
s:SMS_PROG:ECF_PORT:g
s:SMSINCLUDE:ECF_INCLUDE:g
s:SMSFILES:ECF_FILES:g
s:SMSTRYNO:ECF_TRYNO:g
s:SMSTRIES:ECF_TRIES:g
s:SMSHOME:ECF_HOME:g
s:SMSRID:ECF_RID:g
s:SMSJOB:ECF_JOB:g
s:SMSJOBOUT:ECF_JOBOUT:g
s:SMSOUT:ECF_OUT:g
s:SMSCHECKOLD:ECF_CHECKOLD:g
s:SMSCHECK:ECF_CHECK:g
s:SMSLOG:ECF_LOG:g
s:SMSLISTS:ECF_LISTS:g
s:SMSPASSWD:ECF_PASSWD:g
s:SMSSERVERS:ECF_SERVERS:g
s:SMSMICRO:ECF_MICRO:g
s:SMSPID:ECF_PID:g
s:SMSHOST:ECF_HOST:g
s:SMSDATE:ECF_DATE:g
s:SMSURL:ECF_URL:g
s:SMSURLBASE:ECF_URLBASE:g
s:SMSCMD:ECF_JOB_CMD:g
s:SMSKILL:ECF_KILL_CMD:g
s:SMSSTATUSCMD:ECF_STATUS_CMD:g
s:SMSURLCMD:ECF_URL_CMD:g
s:SMSWEBACCESS:ECF_WEBACCESS:g
s:SMS_VERS:ECF_VERS:g
s:SMS_VERSION:ECF_VERSION:g
/edit ECF_INCLUDE/ {
s:/include:/include_ecf:g
}
/edit ECF_INCLUDE/ {
s:_prod:_prod_ecf:g
}
/edit ECF_FILES/ {
s:_prod:_prod_ecf:g
}
s:smshostfile:ecf_hostfile:g
s:sms_hosts:ecf_hosts:g

</pre>
</div>
<p>Applying such a filter to all sms tasks can be simplfied:</p>
<div class="highlight-python"><pre>#!/bin/ksh
files=`find -type f -name "*.sms"  ## all sms wrappers
for f in $files ; do
ecf=$(basename $f .sms).ecf        ## ecf task name
sed -f sms2ecf-min.sed $f &gt; $ecf   ## translate
diff $f $ecf &gt; /dev/null &amp;&amp; rm $ecf &amp;&amp; ln -sf $f $g ## or link
done</pre>
</div>
<p>SMS wrappers links can be preserved:</p>
<div class="highlight-python"><pre>#!/bin/ksh
files=`find -type l -name "*.sms" `
for f in $files ; do
ecf=$(basename $f .sms).ecf        ## ecf task name
link=$(readlink $f)
dir=$(dirname $f); cd $dir
ln -sf $link $ecf
cd -
done</pre>
</div>
<p>Special attention is needed for the variables renaming:</p>
<table border="1" class="docutils">
<colgroup>
<col width="48%" />
<col width="52%" />
</colgroup>
<thead valign="bottom">
<tr class="row-odd"><th class="head"><p class="first last">SMS</p>
</th>
<th class="head"><p class="first last">ecFlow</p>
</th>
</tr>
</thead>
<tbody valign="top">
<tr class="row-even"><td><p class="first last">SMSCMD</p>
</td>
<td><p class="first last">ECF_JOB_CMD</p>
</td>
</tr>
<tr class="row-odd"><td><p class="first last">SMSKILL</p>
</td>
<td><p class="first last">ECF_KILL_CMD</p>
</td>
</tr>
<tr class="row-even"><td><p class="first last">SMS_STATUSCMD</p>
</td>
<td><p class="first last">ECF_STATUS_CMD</p>
</td>
</tr>
<tr class="row-odd"><td><p class="first last">SMS_URLCMD</p>
</td>
<td><p class="first last">ECF_URL_CMD</p>
</td>
</tr>
</tbody>
</table>
<p>It is not a good idea to systematically replace SMS with ECF_, for
example, we use the variables NO_SMS and LSMSSIG which are not related to SMS.</p>
</li>
<li><p class="first">If we want to run the the same job using both SMS and ecFlow,
%SMSXXX% may be replaced with shell variables ECF_XXX. Then in a
header file, we will define ECF_XXX=%SMSXXX:0% for sms mode and
ECF_XXX=%ECF_XXX:0% for ecFlow mode.</p>
</li>
<li><p class="first">All tasks calling CDP directly must be treated carefully and text
client commands replaced with their ecFlow counterpart. They may
force complete a family or a task, requeue a job or change a variable value:</p>
<div class="highlight-python"><pre>#!/usr/bin/env cdp
cdp &lt;&lt; EOF
define ERROR {
  if(rc==0) then exit 1; endif
}

set SMS_PROG %SMS_PROG%
login %SMSNODE% %USER% 1 ; ERROR
suites -s %SUITE%
loop task ( $missing ) do
  force -r complete /%SUITE%/%FAMILY%/tc\$task ; ERROR
endloop
exit
EOF</pre>
</div>
<p>The ECF_PORT variable gives us the ability to discriminate between jobs under
ecFlow control or not:</p>
<div class="highlight-python"><pre>#!/bin/ksh

if [ %ECF_PORT:0% -gt 0 ] ; then
  for task in $missing; do
    ecflow_client --force complete recursive /%SUITE%/%FAMILY%/tc$task
  done
else

cdp &lt;&lt; EOF
define ERROR {
  if(rc==0) then exit 1; endif
}

set SMS_PROG %SMS_PROG%
login %SMSNODE% %USER% 1 ; ERROR
suites -s %SUITE%
loop task ( $missing ) do
  force -r complete /%SUITE%/%FAMILY%/tc\$task ; ERROR
endloop
exit
EOF
fi
</pre>
</div>
</li>
<li><p class="first">sms child commands may also be called in few sms task wrappers.  These should again
be replaced with their ecFlow equivalents.</p>
</li>
</ul>
</div>