ecFlow's documentation is now on readthedocs!

Currently, in ecFlow, we can have jobs that are identical but vary only in the step.

In these cases, each step carries a latency, i.e. submission  cost,(i.e. starting hundreds of new processes)

The Queue attribute was created to address this situation.

suite test
  family f1
     task 001
     task 002
     task 003
     task 004
     task 005
  endfamily
endsuite
suite test
  family f1
     queue q1 001 002 003 004 005 006
     task t
  endfamily
endsuite
suite test_queue
  family f1
     queue q1 001 002 003 004 005 006 007
     task t
  endfamily
  family f2
     queue q2 1 2 3 4 5 6 8 9 10
     task a
     task b
        # notice that queue name is accessible to the trigger
        trigger /test_queue/f1:q1 > 5      
     task c
        trigger ../f2/a:q2 > 9
  endfamily
endsuite

There is a new child command --queue, that will signal when a step is active, complete, or has aborted.

New Child command, for use in .ecf scripts
# Note: because --queue is treated like a child command(init,complete,event,label,meter,abort,wait), the task path ECF_NAME is read from the environment

# The --queue command will search up the node hierarchy for the queue name. If not found it fails.

step=$(ecflow_client --queue queue_name  active)                # returns first queued/aborted step from the server and makes it active, Return "NULL" for the last step.
ecflow_client --queue queue_name complete $step                 # Tell the server that step has completed for the given queue
ecflow_client --queue queue_name aborted  $step                 # Tell the server that step has aborted for the given queue
no_of_aborted=$(ecflow_client --queue queue_name no_of_aborted) # returns as a string the number of aborted steps 
ecflow_client --queue queue_name reset                          # sets the index to the first queued/aborted step. Allows steps to be reprocessed for errors

This attribute makes it possible to follow a producer(server)/consumer(tasks) pattern. Note additional task consumers can be added for load balancing.

The queue values can be strings, however, if they are to be used in trigger expressions, they must be convertible to integers.

suite test_queue
      family f1
          queue q1 red orange yellow green blue indigo violet
          task t
      endfamily
endsuite