A meter can be used to monitor the disk space available in the working directory of the suite's tasks. This will enable the possibility to postpone the execution of space demanding tasks by defining a trigger condition on that meter. In this simple pattern, a check_disk family contains the meter (named use) while an update_disk task runs periodically, by cron, to modify the value of the meter.
Family definition from ecf import Family,Meter,Task,Cron, ... ... Family("check_disk").add( Meter("use",-1,100), Task("update_disk").add( Cron("00:00 23:59 00:05")) ... | Task definition #!/usr/bin/ksh %manual DESCRIPTION Check available disk space in working directory's fileset and update meter %end %include <init.h> value=$(df %WORKDIR% | grep %WORKDIR% | awk '{print $5}' | sed s/%%//g) ecflow_client --alter change meter "use" "$value" /%SUITE%/%FAMILY% %include <endt.h> |
This pattern has been used in a a suite that reads re-analysis data from mars and re-archives it with a different experiment version number. This sort of processing is inherent parallel and, quite often, the main constraint that limits its parallelism is the amount of disk space available for files that have to be created to temporary store the data.