A meter can be used to monitor the disk space available in the working directory of the suite's tasks. This will enable the possibility to postpone the execution of space-demanding tasks by defining a trigger condition on that meter. In this simple pattern, a check_disk family contains the meter (named use) while an update_disk task runs periodically, by cron, to modify the value of the meter.
Code Block | ||||
---|---|---|---|---|
| ||||
from ecf import Family,Meter,Task,Cron, ... ... Family("check_disk").add( Meter("use",-1,100), Task("update_disk").add( Cron("00:00 23:59 00:05")) ... |
Code Block | ||||
---|---|---|---|---|
| ||||
#!/usr/bin/ksh %manual DESCRIPTION Check available disk space in working directory's fileset and update meter %end %include <init.h> value=$(df %WORKDIR% | grep %WORKDIR% | awk '{print $5}' | sed s/%%//g) ecflow_client --alter =change meter "use" "$value" /%SUITE%/%FAMILY% %include <endt.h> |
This pattern has been used, for instance, in a suite that reads re-analysis data from mars and re-archives it with a different experiment version number. This sort of processing is inherent parallel but, quite often, it requires the creation of files to temporarily store the data. So it is a commom common case that the amount of disk space available for those files is the main constraint that limits the parallelism of the process.
Gliffy Diagram | ||
---|---|---|
|