- Created by Daniel Varela Santoalla, last modified by Axel Bonet on Nov 08, 2021
ecFlow is a client/server workflow package that enables users to run a large number of programs (with dependencies on each other and on time) in a controlled environment. It provides reasonable tolerance for hardware and software failures, combined with restart capabilities. It is used at ECMWF to run all our operational suites across a range of platforms.
ecFlow submits tasks (jobs) and receives acknowledgments from the tasks when they change status and when they send events, using child commands embedded in your scripts. ecFlow stores the relationships between tasks, and is able to submit tasks dependant on triggers, such as when a given task changes its status, for example when it finishes.
ecFlow functionality is provided by the following executables and shared libraries
- ecflow_client: This executable is a command-line program; it is used for all communication with the server. This executable implements the Command Level Interface (CLI). The bulk of this functionality is also provided by the python API
- ecflow_ui: This is a specialized GUI client that monitors and visualizes the node tree hierarchy. Based on QT.
- ecflow_server: This executable is the server. It is responsible for scheduling the jobs and responding to ecflow_client requests.
- ecflow.so: Python interface These shared libraries provide the python API for creating the suite definition and communication with the server.
ecFlow runs as a server receiving requests from clients. CLI, GUI, and suite jobs are clients. Communication is based on TCP/IP. Note that ecFlow is a scheduler and is not a queuing system such as NQS, SGE, Load leveler, or PBS. However, it can submit to queuing systems.
History
For almost three decades ECMWF used SMS (Supervisor Monitoring Scheduler) package to control the workflow for the Centre's operational models and systems. Written at ECMWF, it allows the design, submission, and monitoring of jobs both in the Research and Operations Departments, and provides common tools for scientists, analysts, and operators to cooperate. A large number of organizations also use SMS (from both Member and non-Member States). Development and support of SMS has now stopped.
The replacement, ecFlow 4, has superseded SMS. It has been used for over a decade. It is a complete rewrite using an object-oriented methodology and modern standardized components; it acts as a comprehensive replacement for SMS. The rewrite helps improve maintainability, allows easier modification, and introduces object orientated features. The proprietary script language used by SMS, CDP, has been replaced by Python.
The development of ecflow 4 has now stopped. But support is still provided. It has been replaced by ecflow 5
You can find an introduction to ecFlow in the ECMWF Newsletter article (Autumn 2011), starting on page 30.
-
Page:ecflow release 5.9.0 (ecFlow)
-
Page:ecflow release 5.8.3 (ecFlow)
-
Page:ecflow release 5.8.1 (ecFlow)
-
Page:ecflow release 5.8.0 (ecFlow)
-
Page:ecflow release 5.7.0 (ecFlow)
Go to Latest news to access older news items
Please send an email to software.support@ecmwf.int or go to Issues if you have any suggestions for improvements or have discovered a bug with this software package.
Blog Posts
-
Blog: ecflow standalone GUIs
created by
Jan 22, 2014
-
Blog: ecflow watchdogs
created by
Jan 21, 2014
File | Modified | |
---|---|---|
ZIP Archive ecFlow-5.12.0-Source.tar.gz | Feb 13, 2024 by Marcos Bento | |
Labels
|
||
ZIP Archive ecFlow-5.11.4-Source.tar.gz | Oct 12, 2023 by Iain Russell | |
Labels
|
||
ZIP Archive ecFlow-5.11.3-Source.tar.gz | Jul 18, 2023 by Iain Russell | |
Labels
|
||
ZIP Archive ecFlow-5.11.1-Source.tar.gz | Jul 04, 2023 by Iain Russell | |
Labels
|
||
ZIP Archive ecFlow-5.11.0-Source.tar.gz | May 25, 2023 by Iain Russell | |
Labels
|
||
ZIP Archive ecFlow-5.10.0-Source.tar.gz | Feb 09, 2023 by Iain Russell | |
Labels
|
||
File ecFlow-5.9.2-Source.tar.gz | Nov 30, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.9.1-Source.tar.gz | Nov 10, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.9.0-Source.tar.gz | Oct 07, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.8.4-Source.tar.gz | May 14, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.8.3-Source.tar.gz | Mar 08, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.8.1-Source.tar.gz | Jan 27, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.8.0-Source.tar.gz | Jan 26, 2022 by Application deploy | |
Labels
|
||
File ecFlow-5.7.2-Source.tar.gz Hotfix release | Sep 17, 2021 by Application deploy | |
Labels
|
||
File ecFlow-5.7.1-Source.tar.gz | Sep 13, 2021 by Application deploy | |
Labels
|
||
File ecFlow-5.7.0-Source.tar.gz | May 19, 2021 by Application deploy | |
Labels
|
||
File ecFlow-5.6.0-Source.tar.gz | Nov 25, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.5.3-Source.tar.gz | Sep 24, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.5.2-Source.tar.gz | Sep 10, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.5.1-Source.tar.gz | Jul 09, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.5.0-Source.tar.gz | Jul 03, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.4.0-Source.tar.gz | May 06, 2020 by Application deploy | |
Labels
|
||
ZIP Archive ecFlow-5.3.1-Source.tar.gz production release | Apr 02, 2020 by Avi Bahra | |
Labels |
||
File ecFlow-4.17.2-Source.tar.gz production release | Mar 03, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.3.0-Source.tar.gz production release | Mar 03, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.2.3-Source.tar.gz production release | Feb 18, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.2.2-Source.tar.gz production release | Feb 14, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.2.1-Source.tar.gz production release | Jan 29, 2020 by Application deploy | |
Labels
|
||
File ecFlow-5.2.0-Source.tar.gz production release | Jan 20, 2020 by Application deploy | |
Labels
|
||
File ecFlow-4.17.1-Source.tar.gz production release | Jan 08, 2020 by Application deploy | |
Labels
|
||
File ecFlow-4.17.0-Source.tar.gz production release | Oct 10, 2019 by Application deploy | |
Labels
|
||
File ecFlow-5.1.0-Source.tar.gz production release | Oct 02, 2019 by Application deploy | |
Labels
|
||
File ecFlow-4.16.0-Source.tar.gz production release | Aug 15, 2019 by Application deploy | |
Labels
|
||
File ecFlow-4.15.0-Source.tar.gz production release | Jun 24, 2019 by Application deploy | |
Labels
|
||
File ecFlow-4.14.0-Source.tar.gz production release | Mar 27, 2019 by Application deploy | |
Labels |
||
File ecFlow-4.13.0-Source.tar.gz production release | Feb 21, 2019 by Application deploy | |
Labels
|
||
File ecFlow-4.12.0-Source.tar.gz production release | Jan 17, 2019 by Application deploy | |
Labels
|
||
File ecFlow-4.11.1-Source.tar.gz production release | Oct 23, 2018 by Application deploy | |
Labels
|
||
File ecFlow-4.11.0-Source.tar.gz production release | Oct 16, 2018 by Application deploy | |
Labels |
||
ZIP Archive ecFlow-4.10.0-Source.tar.gz production release | Jul 23, 2018 by Avi Bahra | |
Labels |
||
File ecFlow-4.9.0-Source.tar.gz production release | Apr 05, 2018 by Application deploy | |
Labels |
||
ZIP Archive ecFlow-4.8.0-Source.tar.gz production release | Jan 16, 2018 by Avi Bahra | |
Labels |
||
ZIP Archive ecFlow-4.7.1-Source.tar.gz production release(hotfix) | Nov 07, 2017 by Avi Bahra | |
Labels |
||
ZIP Archive ecFlow-4.7.0-Source.tar.gz production release | Oct 11, 2017 by Avi Bahra | |
Labels |
||
ZIP Archive boost_1_53_0.tar.gz Boost libs for ecflow > 4.0.0 | Nov 14, 2013 by Avi Bahra | |
Labels
|
Recently Updated
-
ecFlow-5.12.0-Source.tar.gzFeb 13, 2024 • attached by Marcos Bento
-
ecFlow White list fileJan 26, 2024 • commented by Bojan Kasic
-
ecFlow White list fileJan 26, 2024 • commented by Christopher Roberts
-
ecFlow-5.11.4-Source.tar.gzOct 12, 2023 • attached by Iain Russell
-
-
-
ecFlow-5.11.3-Source.tar.gzJul 18, 2023 • attached by Iain Russell
-
ecFlow-5.11.1-Source.tar.gzJul 04, 2023 • attached by Iain Russell
-
ecFlow-5.11.0-Source.tar.gzMay 25, 2023 • attached by Iain Russell
-
-
-
ecFlow-5.10.0-Source.tar.gzFeb 09, 2023 • attached by Iain Russell
-
-
ecFlow-5.9.2-Source.tar.gzNov 30, 2022 • attached by Application deploy
-
ecFlow-5.9.1-Source.tar.gzNov 10, 2022 • attached by Application deploy