Extending woom

Extending woom#

One can extend the capabilities of woom by supplying specific files.

Jinja filters#

Jina is used to perform string substitutions to generate the task scripts to be submitted as jobs. This process supports filtering that transforms you data after converting it to string. Jinja builtin filters are available here. Woom provides a few filters in the woom.render module.

You can add you own filter functions with the following procedure:

  1. Create a python file named jinja_filters.py located in the ext/ folder of your workflow directory.

  2. Declare in this file all the jinja filter functions you want.

  3. Declare a dictionary named JINJA_FILTERS to register your functions.

Example:

examples/academic/ensemble/ext/jinja_filters.py#
import string


def filter_member2letter(member):
    """Convert a int to an uppercase letter"""
    return string.ascii_uppercase[member.id - 1]


JINJA_FILTERS = {"member2letter": filter_member2letter}

If present, this file is loaded at workflow setup by the woom.ext.load_jinja_filters() function.

Jinja templates#

You can extend the default jinja templates by providing yours in the templates directory of the workflow directory.

Examples:

examples/academic/templates/templates/env.sh#
{% extends "!env.sh" %}

{% block env_vars -%}
{{ super() }}
export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK:-4}
{% endblock %}

{% block custom %}
# Utilities
function log_message() {
    echo "[$(date +%Y-%m-%d_%H:%M:%S)] $*" | \
        tee -a ${WOOM_LOG_DIR}/custom.log
}

# Initialization
log_message "Environment is initialized for ${WOOM_TASK_NAME}"
{% endblock %}
examples/academic/templates/templates/job.sh#
{% extends "!job.sh" %}

{% block header %}
{{ super() }}
# Workflow: {{ app_name|default('N/A') }}
# Configuration: {{ app_conf|default('N/A') }}
# Experiment: {{ app_exp|default('N/A') }}
# Task: {{ task.name }}
{% endblock %}

See Use the templating system.

Configobj specifications#

Parameters that are passed to the workflow configuration in the [params] are interpreted as strings by default. You can change this behavior by changing adding your own specification file, and optionally by providing new validator functions that convert text to the desired type.

Workflow configuration specifications#

Just pass the commandline option –workflow-ini to give a configuration specification file that will be merged with the default one. For info about how to write such file, refer to the doc. An specification option can refer to an existing validator function like boolean or a new one (see next section).

Example:

examples/academic/ensemble/workflow.ini#
[ensemble]
    [[iters]]
    ks=random_lognormal(default=None)

Validator functions#

You can add new validator functions by following the procedure:

  1. Create a python file named validator_functions.py located in the ext/ folder of your workflow directory.

  2. Declare in this file all the validator functions you want.

  3. Declare a dictionary named VALIDATOR_FUNCTIONS to register your functions.

Example:

examples/academic/ensemble/ext/validator_functions.py#
import random


def random_lognormal(specs):
    if str(specs) == "None":
        return
    mu = float(specs[0])
    sigma = float(specs[1])
    size = int(specs[2])
    return [random.lognormvariate(mu, sigma) for i in range(size)]


VALIDATOR_FUNCTIONS = {"random_lognormal": random_lognormal}

If present, this file is loaded at workflow setup by the woom.ext.load_validator_functions() function.

Contents