From python

Instead of calling the commandline interface, you can run your workflow from python. Here is a minimal list of operations you can do. Have a look to the woom.cli module for more info.

Initializations

Import modules:

import logging
import woom.log as wlog
import woom.ext as wext
import woom.tasks as wtasks
import woom.hosts as whosts
import woom.workflow as wworkflow

Initialize the logger:

wlog.setup_logger(console_level="debug", to_file="woom.log")
logger = logging.getLogger("woom")

Setup the workflow

Load the extension files with load_jinja_filters() and load_validator_functions():

wext.load_jinja_filters("ext/jinja_filters.py") # Jinja filters
wext.load_validator_functions("ext/validator_functions.py") # Configobj validator functions

Alternatively, if your extension files are in the ext/ directory, you can use load_extensions():

wext.load_extensions(".")

Initialize the host manager:

hostmanager = whosts.HostManager()
hostmanager.load_config("hosts.cfg")

Set your host:

host = hostmanager.get_host("datarmor")

Alternatively, you can infer your host thanks to the patterns configuration option if set:

host = hostmanager.infer_host()

Initialize the task manager:

taskmanager = wtasks.TaskManager(host)
taskmanager.load_config("tasks.cfg")

Finally, initialize the workflow:

workflow = wworkflow.Workflow("workflow.cfg", taskmanager)

Operate

To print an overview, call get_overview():

workflow.show_overview()

If it’s ok, run the workflow with run(). Do it first in fake mode so that it will tell what will be done without doing it:

workflow.run(dry=True)

The run it for real:

workflow.run()

If you just want to re-run tasks that have already run, use the update keyword:

workflow.run(update=True)

To check the status of all workflow tasks and their associated jobs, use show_status():

workflow.show_status()

To show the status of only running jobs, use the running keyword:

workflow.show_status(running=True)

If you wan to kill all your jobs, call kill():

workflow.kill()

You can fine tune which jobs you want to kill:

workflow.kill("1256")                                          # by job id
workflow.kill(task_name="mytask")                              # by task
workflow.kill(cycle="2020-01-01T00:00:00-2020-01-01T06:00:00") # by cycle
workflow.kill(member=1)                                        # by ensemble member id

Filling templates

If your tasks have a [[fill]] configuration section, you can manually fill templates using fill_templates().

First, set the context for the task you want to fill templates for:

workflow.set_context(task_name="run_model", cycle="2020-01-01T00:00:00", member=1)

Then fill all templates configured for this task:

workflow.fill_templates()

This will render all templates defined in the task’s [[fill]] section and write them to their destination paths.

Alternatively, you can fill a single template directly:

task = workflow.get_task("run_model")
task.set_context(workflow.get_context(task_name="run_model", cycle="2020-01-01T00:00:00"))
task.fill_templates()

See Templating In-Depth for more information about user templates and the fill feature.