SMS triggers check are from the client side only,
ie under care of the user. It is simple to reach a situation where,
from the server side, triggers are no more valid, and related task remains queued (deadlock).
To prevent that, ecFlow triggers check is more robust:
- triggers have to be consistent from the client side.
- 'extern' can only refer to an external suite's node
- server can be requested to confirm that all triggers are referring to an existing node,
in already loaded suites.
Like the SMS, a ecFlow suite can be used for team work. It is suite design and management skill to decide
how to split the suite for a good collaboration within team members.
At ECMWF, operational suites are defined in files describing the full content.
- the definition file is split into different python modules for
- observations retrieval
- analysis and forecast
- products generation and dissemination
- plots generation
- then a python script is in charge to put all things together and to create the suite
- the user decides to load the full suite, only a sub-tree from the suite or to load it as an e-suite (test suite)
- a SCM is used to identify the modules edited by different users, merge their updates, resolve the conflicts or allow them to work in an branched version or the suite for the next release.
The following example shows that with ecFlow
- a master definition file may provide the suite structure (multi1.def), with the right setting of inlimits, limits, triggers,
- a developer in charge of adding test task to the suite may update it with a task test2, below an agreed family (/multi/main/00/test),
- another tester may add some other tests as a different suite, triggered from 'multi' suite.
ecflow_client --load multi1.def
ecflow_client --replace /multi/main/00/test/fc_complete/test2 multi2.def
ecflow_client --replace /test20121123 multi3.def
python TestBench.py multi1.def
python TestBench.py multi3.def
suite limits
defstatus complete
limit power6 120
limit power7 120
limit prod 10
limit diss 10
limit test 10
endsuite
suite multi
family main
inlimit /limits:power6
repeat date YMD 20120101 20301212
family 00
family obs;
task get;
endfamily
family ana
trigger obs eq complete
task ifs
endfamily
family fc
trigger ana eq complete
task model
meter step -1 120
endfamily
family prod
inlimit /limits:prod
family 000;
trigger /multi/main/00/fc/model:step gt 0 or /multi/main/00/fc/model eq complete
task gen; endfamily
family 012;
trigger /multi/main/00/fc/model:step gt 12 or /multi/main/00/fc/model eq complete
task gen; endfamily
family 024;
trigger /multi/main/00/fc/model:step gt 24 or /multi/main/00/fc/model eq complete
task gen; endfamily
family 048;
trigger /multi/main/00/fc/model:step gt 48 or /multi/main/00/fc/model eq complete
task gen; endfamily
family 096;
trigger /multi/main/00/fc/model:step gt 96 or /multi/main/00/fc/model eq complete
task gen; endfamily
family 120;
trigger /multi/main/00/fc/model:step gt 120 or /multi/main/00/fc/model eq complete
task gen; endfamily
endfamily prod
family diss
inlimit /limits:diss
family 000;
trigger /multi/main/00/prod/000 eq complete
task gen; endfamily
family 012;
trigger /multi/main/00/prod/012 eq complete
task gen; endfamily
family 024;
trigger /multi/main/00/prod/024 eq complete
task gen; endfamily
family 048;
trigger /multi/main/00/prod/048 eq complete
task gen; endfamily
family 096;
trigger /multi/main/00/prod/096 eq complete
task gen; endfamily
family 120;
trigger /multi/main/00/prod/120 eq complete
task gen; endfamily
endfamily diss
family test
inlimit /limits:test
family fc_complete
trigger /multi/main/00/fc eq complete
task test1
endfamily
family prod_complete
trigger /multi/main/00/prod eq complete
task test1
endfamily
family diss
trigger /multi/main/00/diss eq complete
task test1
endfamily
endfamily
endsuite
extern /limits:test
suite multi
defstatus suspended
family main
family 00
family test
family fc_complete
task test2
family fam_test
trigger test2 eq complete
task prepare
task process
trigger prepare eq complete
task push
trigger process eq complete
extern /limits:test
extern /multi/main/00/fc/model:step
extern /multi/main:YMD
suite test20121123
defstatus suspended # so that it can be changed manually to queued
repeat date YMD 20120101 20301212
family test
repeat integer STEP 0 120
trigger /multi/main:YMD gt /test20121123:YMD or (/multi/main:YMD eq /test20121123:YMD and (/multi/main/00/fc/model:step ge /test20121123/test:STEP or /multi/main/00/fc/model eq complete))
task run
