Page History

<div class="section" id="introduction">
<span id="migration-intro"></span><span id="index-0"></span>
<p>This document describes the main steps in moving from as SMS system to a suite
managed by an ecFlow server. Related documentation for each system can be found here:</p>
<ul class="simple">
<li><a class="reference external" href="http://www.ecmwf.int/publications/manuals/sms/">SMS</a>.</li>
<li><a class="reference external" href="/wiki/display/ECFLOW/ecflowhttp://intra.ecmwf.int/metapps/manuals/ecflow/">ecFlow</a>.</li>
</ul>
<p>Multiple scenarios are available:</p>
<ul>
<li><p class="first">the best case is perhaps when starting a new project, with no prior
historical constraint nor inherited scripts. A suite designer can
choose their preferred language for the suite definition, either
python, or using another any language to produce a text definition file such as
simple ksh scripting, other scripting languages, an interpreted or a compiled language.
The best language is the one you are fluent with, pleased to work with,
alone and as a team. Its purpose is to produce an expanded definition file,
containing the description of the tasks to run, their time, date and trigger
dependencies, their specific configuration (variables). For SMS, the play environment
keywords available are (CDP&gt;man -s play):</p>
<div class="highlight-python"><pre>EOF          abort        action       autocancel   automigrate  autorestore
clock        complete     cron         date         day          defstatus
edit         endfamily    endsuite     endtask      event        extern
family       inlimit      label        late         limit        meter
owner        repeat       suite        task         text         time
today        trigger</pre>
</div>
<p>For ecFlow this list is slightly smaller:</p>
<div class="highlight-python"><pre>                                       autocancel
clock        complete     cron         date         day          defstatus
edit         endfamily    endsuite     endtask      event        extern
family       inlimit      label        late         limit        meter
             repeat       suite        task                      time
today        trigger</pre>
</div>
<p>Next comes the task wrappers definition. Again, there is no
right way to do this. It is simple to design a task
whose language is pure python or pure perl. We tend to use use Ksh scripting for task
templates for the following reasons:</p>
<blockquote>
<div><ul class="simple">
<li>trap ERROR 0: to prevent early exit from the script and call the ERROR if exited</li>
<li>set -e: to raise an error if a command exit status is not 0</li>
<li>set -u: to prevent undefined variable usage</li>
<li>set -x: to display each command before execution</li>
<li>PS4 variable: to allow time stamping and evaluate each lines runtime</li>
<li>trap: to redirect internal/external signal reception to an ERROR function</li>
</ul>
</div></blockquote>
<p>Task headers can be used to make common what can be shared among multiple
tasks (head.h, tail.h, trap.h, rcp.h, qsub.h).</p>
</li>
<li><p class="first">another migration strategy is a one step migration: this document can be used as a
check-list. However, it may not be exhaustive and may be enriched with your contribution,
reporting a case not mentioned.</p>
</li>
<li><p class="first">a frequent migration strategy may be to evolve to a situation where it is possible to
run a suite under both SMS or ecFlow.</p>
</li>
</ul>
<p>Milestone are:</p>
<ul class="simple">
<li>upgrading header files.</li>
<li>modifying task wrapper files.</li>
<li>writing a definition file that describes the expected suite.</li>
<li>third party software may have migration concerns:<ul>
<li>the IFS model is known to call smsevent, smsmeter,</li>
<li>our archive system MARS transmits information to sms using smslabel, smsevent.</li>
<li>hopefully, these calls are through a system commands. However SMS extensions, linked with the
sms library, are obsolete with ecFlow and cannot be simply migrated.</li>
</ul>
</li>
<li>when both SMS/ecFlow systems have to run in parellel, we have to
decide what is part of the suite configuration (user side), and what
is part of the system responsibility.</li>
<li>on the system side:<ul>
<li>smssubmit, smskill, smsstatus: have to decide if the same script
can be used, if their name has to be changed or if we take this opportunity
to modify their behaviour.</li>
<li>child command calls (smsinit, smscomplete, smsabort ...) can be intercepted by
changing the PATH variable in order to use the same shell scripts that can
call an SMS child command if SMS_PROG is part of the environment or an
ecFlow child command when ECF_PORT is defined and not null.</li>
<li>When the link between a job and its parent server cannot be
established, the file $HOME/.smshostfile is used by child
commands to identify potential backup servers to contact (network
zombie). It has to be decided if the ecFlow servers deployment needs to
match exactly what was chosen for SMS.</li>
</ul>
</li>
</ul>
</div>