<div class="section" id="example">
<span id="index-0"></span><span id="id1"></span>
<p>A suite template is provided to demonstrate basic SMS features which
can be used as a starting point for suite design.</p>
<ul>
<li><p class="first">Download the tar-file <a class="reference download internal" href="../../_downloads/skull.tgz"><tt class="xref download docutils literal"><span class="pre">skull</span> <span class="pre">suite</span></tt></a></p>
</li>
<li><p class="first">untar it:</p>
<div class="highlight-python"><pre>> tar -xzvf skull.tgz</pre>
</div>
</li>
<li><p class="first">eventually, start a SMS server:</p>
<div class="highlight-python"><pre>> sms_start</pre>
</div>
</li>
<li><p class="first">convention at ECMWF is to attribute a program number
according to the formula:</p>
<div class="highlight-python"><pre>> prog_num=$((900000 + $(id -u))); echo $prog_num</pre>
</div>
</li>
<li><p class="first">open xcdp, File->Login to: “localhost <prog_num>”</p>
</li>
<li><p class="first">play the skull suite into sms:</p>
<div class="highlight-python"><pre>> export SMS_PROG=$((900000+$(id -u)))
> LOGGER="setenv -i $SMS_PROG; login localhost $USER 1;"
> /usr/local/apps/sms/bin/cdp -c "$LOGGER play -r /skull skull.def"</pre>
</div>
</li>
<li><p class="first">skull suite families are</p>
<ul class="simple">
<li>make: to open a xterm window from jobs run on local workstation, ecgate, HPC, linux cluter</li>
<li>consumer: to demonstrate the producer-consumer pattern</li>
<li>dinner: a resource sharing example through the dinning philosophers example</li>
<li>shop: to practice dynamic tasks generation</li>
<li>perl, Python: as task template examples that do not use ksh</li>
</ul>
<img alt="../../_images/skull.png" src="/wiki/download/attachments/7373012/skull.png" />
</li>
<li><p class="first">conversion to a text based definition file written by ksh (skull.sh) is rather straightfoward:</p>
<div class="highlight-python"><pre>> make sh</pre>
</div>
<p>libgen.sh is generated to provide cdp:play commands in ksh environment:</p>
<div class="highlight-python"><pre>> bin/genlibdef.sh sh libgen.sh</pre>
</div>
<div class="highlight-python"><pre>set -eu
if [[ ${DEBUG_DEF2ECF:=0} != 0 ]] ; then set -eux ; fi
alias EOF='echo EOF >&3 '
alias abort='echo abort >&3 '
alias action='echo action >&3 '
alias autocancel='echo autocancel >&3 '
alias automigrate='echo automigrate >&3 '
alias autorestore='echo autorestore >&3 '
alias clock='echo clock >&3 '
alias complete='echo complete >&3 '
alias cron='echo cron >&3 '
alias date='echo date >&3 '
alias day='echo day >&3 '
alias defstatus='echo defstatus >&3 '
alias edit='echo edit >&3 '
alias endfamily='echo endfamily >&3 '
alias endsuite='echo endsuite >&3 '
alias endtask='echo endtask >&3 '
alias event='echo event >&3 '
alias extern='echo extern >&3 '
alias family='echo family >&3 '
alias inlimit='echo inlimit >&3 '
alias label='echo label >&3 '
alias late='echo late >&3 '
alias limit='echo limit >&3 '
alias meter='echo meter >&3 '
alias owner='echo owner >&3 '
alias repeat='echo repeat >&3 '
alias suite='echo suite >&3 '
alias task='echo task >&3 '
alias text='echo text >&3 '
alias time='echo time >&3 '
alias today='echo today >&3 '
alias trigger='echo trigger >&3 '
</pre>
</div>
<p>At this stage, libgen.sh can be manually updated, to replace EOF
with exit 0, inhibit the action, automigrate, autorestore, owner commands
(which are unavailable in ecFlow). Change the limit, date, time naming, as it
interferes with ksh intrinsics.</p>
<p>It is possible to transcode most CDP commands into ksh, defining
aliases that will print the cdp:play command and redirect their output in a
dedicated file descriptor:</p>
<div class="highlight-python"><pre>> bin/def2ecf.sh sh gen</pre>
</div>
<p>sh/skull.sh contains the ksh command to create the file descriptor 3:</p>
<div class="highlight-python"><pre>> exec 3> expanded.tmp</pre>
</div>
<p>SMS <strong>.def</strong> definition files may be updated to facilitate such
transcoding (#CONV tags into *.def files):</p>
<blockquote>
<div><ul>
<li><p class="first">commands can be split into multiline commands to simplify
mathematical expressions
incompatible commands (owner, autorestore, automigrate, action)
can be removed or set as a comment</p>
</li>
<li><p class="first">multiline triggers changed into one line commands</p>
</li>
<li><p class="first">the heavy use of aliases can lead to confused transcoding</p>
</li>
<li><p class="first">CDP is known to be “tolerant”: for example, an unboumded endin
command is transparent to CDP, but it would mislead the
transcoder.</p>
</li>
<li><p class="first">for CDP, commands accept minimum significant string for options;
they should be replace with their unambiguous version:</p>
<div class="highlight-python"><pre>> # CDP: repeat int iter 1 $life_expectency
> repeat integer iter 1 $life_expectency</pre>
</div>
</li>
<li><p class="first">through its simplicity, the CDP language was a way to keep away from
complex code. Once left behind, it is each suite designers
responsability to stick to simple manageable, portable and
maintainable code.</p>
</li>
<li><p class="first">beware homonyms: such as date, time, limit</p>
</li>
</ul>
</div></blockquote>
<p>This demonstrates the feasibility, for a simple suite. Past
experience shows it is not as easy when considering a more complex suite design.</p>
<p>The suite definition is then expanded, as a file sh/expanded.tmp:</p>
<div class="highlight-python"><pre>> bin/def2ecf.sh sh expand</pre>
</div>
<p>or:</p>
<div class="highlight-python"><pre>> cd sh; ksh skull.sh</pre>
</div>
</li>
<li><p class="first">the suite can be played locally and expanded for comparison:</p>
<div class="highlight-python"><pre>> /usr/local/apps/sms/bin/cdp -c "play -l skull.def; show > expand.cdp"
> kompare expand.cdp sh/expand.tmp
> kompare skull.def sh/skull.sh</pre>
</div>
</li>
<li><p class="first">it can then be loaded into SMS:</p>
<div class="highlight-python"><pre>> /usr/local/apps/sms/bin/cdp -c "LOGGER play -r /skull sh/expanded.tmp"</pre>
</div>
</li>
<li><p class="first">apply the command begin and resume with XCdp or CDP</p>
</li>
<li><p class="first">We can then start ecFlow server, if not already done:</p>
<div class="highlight-python"><pre>> use ecflow
> export ECF_PORT=$((20000+$(id -u)))
> mkdir ecf && cd ecf
> ecflow_server</pre>
</div>
</li>
<li><p class="first">expanded suite can be loaded into ecFlow:</p>
<div class="highlight-python"><pre>> cd ..
> ecflow_client --load sh/expanded.tmp</pre>
</div>
</li>
<li><p class="first">once loaded, it can be replaced with:</p>
<div class="highlight-python"><pre>> ecflow_client --replace /skull sh/expanded.tmp</pre>
</div>
</li>
<li><p class="first">commands begin and resume can be applied on the console or with ecflowview:</p>
<div class="highlight-python"><pre>> ecflow_client --begin /skull
> ecflow_client --resume /skull
> ecflow_client --resume /skull/skull /skull/perl /skull/python</pre>
</div>
</li>
<li><p class="first">in this example, it is enough to <strong>link task wrappers</strong> from the
original name <em>.sms</em> to the expected name <em>.ecf</em>.</p>
<p>Alternatively, ECF_EXTN variable may be defined, as <strong>.sms</strong> on the
top node.</p>
</li>
<li><p class="first"><strong>smsfiles/passby.sms</strong> task wrapper has been updated to be compatible in both modes.</p>
</li>
<li><p class="first"><strong>smsfiles/perl.sms</strong> and <strong>smsfiles/python.sms</strong> call an intermediate
function to access the meter child command: call_meter</p>
</li>
<li><p class="first">in the directory include: trap.h, perl_header.h, python_header.h,
endt.h, were modified to enable ecFlow mode.</p>
</li>
<li><p class="first"><strong>include/inc_ecf.h</strong> was added and included from trap.h to add new expected variables, when ECF_PORT variable is not 0.</p>
</li>
<li><p class="first">it may be an interesting exercise to transform into Python code the file skull.def and the related families defined in
their dedicated definition files. The script ./bin/def2ecf.pl can be used to produce a transcoded file. Careful attention
is required to fix the transcoder mistakes and obtain a script that can be parsed with Python. Then a script that
can be loaded as a proper suite into SMS, or ecFlow.</p>
<ul class="simple">
<li>pydef_example/inc_hostc.py is a Python library used to maintain a definition file syntax close to CDP’s.</li>
<li>the transcoded file has to be checked for all dollar variables</li>
<li>all CDP functions parameters are transcoded incorrectly: $1, $2 ... They can be transformed into named arguments, with a
default value when needed.</li>
<li>the definition tree is created adding the lines</li>
</ul>
</li>
</ul>
<dl class="docutils">
<dt>::</dt>
<dd><blockquote class="first">
<div>s = suite(SELECTION)
DEFS = Defs()
DEFS.add_suite(s)</div></blockquote>
<ul class="last simple">
<li>choice must be made whether to use “import library” or “from library import *”.</li>
<li>to maintain code readability, new functions can be created (create_suite)</li>
</ul>
</dd>
</dl>
</div>
|