A meter can be used to monitor the disk space available in the working directory of the suite's tasks. This will enable the possibility to postpone the execution of space-demanding tasks by defining a trigger condition on that meter. In this simple pattern, a check_disk family contains the meter (named use) while an update_disk task runs periodically, by cron, to modify the value of the meter.
Code Block | ||||
---|---|---|---|---|
| ||||
from ecf import Family,Meter,Task,Cron, ...
...
|
...
Family("check_disk").add( Meter("use",-1,100), Task("update_disk").add( Cron("00:00 23:59 00:05")) ... |
...
|
Code Block | |||
---|---|---|---|
|
...
| |
#!/usr/bin/ksh
%manual
DESCRIPTION
Check available disk space in working directory's fileset and update meter
%end
%include <init.h>
value=$(df %WORKDIR% | grep %WORKDIR% | awk '{print $5}' | sed s/%%//g)
ecflow_client --alter |
...
=change meter "use" "$value" /%SUITE%/%FAMILY% %include <endt.h> |
This pattern has been used, for instance, in
...
a suite that reads re-analysis data from mars and re-archives it with a different experiment version number. This sort of processing is inherent parallel
...
but, quite often,
...
it requires the creation of files to temporarily store the data. So it is a common case that the amount of disk space available for those files
...
is the main constraint that limits the parallelism of the process.
Gliffy Diagram | ||
---|---|---|
|
...
...