Context and Variables In-Depth#

The context is a dictionary of variables available when rendering templates and configuration files. Understanding what variables are available and how to use them is key to creating flexible, reusable workflows.

What is the Context?#

The context provides information about:

The workflow (application name, paths)
The current task (name, run directory)
The current cycle (dates, tokens)
The current ensemble member (ID, parameters)
Custom parameters from configuration
Environment variables

Access in Templates#

Variables are accessed using Jinja2 syntax:

Task: {{ task.name }}
Cycle: {{ cycle.begin_date }} to {{ cycle.end_date }}
Run directory: {{ task_run_dir }}
Parameter value: {{ params.my_param }}

Access in Python#

From Python code:

context = workflow.get_context(task_name='run_model', cycle='2020-01-01')
print(context['task'].name)
print(context['cycle'].begin_date)
print(context['params']['my_param'])

Workflow Variables#

Application Information#

{{ app.name }}    # Application name
{{ app.conf }}    # Configuration name
{{ app.exp }}     # Experiment name

Example:

Configuration:

[app]
name = ocean_model
conf = north_atlantic
exp = test_2020

Template:

Application: {{ app.name }}
Configuration: {{ app.conf }}
Experiment: {{ app.exp }}

Result:

Application: ocean_model
Configuration: north_atlantic
Experiment: test_2020

Workflow Paths#

{{ workflow_dir }}      # Directory containing workflow.cfg
{{ task_submission_dir }}    # Where job scripts are created
{{ script_path }}       # Path to current job script

Example:

Workflow location: {{ workflow_dir }}
Job script: {{ script_path }}

Task Variables#

Task Information#

{{ task.name }}       # Task name
{{ task_name }}       # Also available (shorthand)
{{ task_path }}       # Full path including app/cycle/task/member
{{ task_run_dir }}         # Where task executes

Example:

Executing task: {{ task.name }}
Task path: {{ task_path }}
Working directory: {{ task_run_dir }}

task_path Components:

For workflow ocean_model/config1/exp1, task run_model, cycle 2020-01-01, member member001:

task_path = ocean_model/config1/exp1/2020-01-01/run_model/member001

Task Object#

Access task configuration:

{{ task.name }}
{{ task.commandline }}
{{ task.run_dir }}

Cycle Variables#

When No Cycles#

If workflow has no cycles or single cycle:

{{ cycle }}           # None or fixed date

When Using Cycles#

Date Information:

{{ cycle.date }}          # Point-in-time date
{{ cycle.begin_date }}    # Interval start
{{ cycle.end_date }}      # Interval end

Formatted Dates:

{{ cycle.date_str }}           # "2020-01-01T00:00:00"
{{ cycle.begin_date_str }}     # "2020-01-01T00:00:00"
{{ cycle.end_date_str }}       # "2020-01-02T00:00:00"

Tokens:

{{ cycle.token }}         # "2020-01-01T00:00:00-2020-01-02T00:00:00"
{{ cycle.label }}         # Same as token

Boolean:

{{ cycle.is_interval }}   # True if interval, False if point

Example - Point Cycle:

Configuration:

[cycles]
begin_date = 2020-06-15
as_intervals = False

Template:

Date: {{ cycle.date }}
Formatted: {{ cycle.date_str }}
Token: {{ cycle.token }}

Result:

Date: 2020-06-15 00:00:00
Formatted: 2020-06-15T00:00:00
Token: 2020-06-15T00:00:00

Example - Interval Cycle:

Configuration:

[cycles]
begin_date = 2020-01-01
freq = 1D
as_intervals = True

Template:

Period: {{ cycle.begin_date }} to {{ cycle.end_date }}
Start: {{ cycle.begin_date_str }}
End: {{ cycle.end_date_str }}
ID: {{ cycle.token }}

Result (for first cycle):

Period: 2020-01-01 00:00:00 to 2020-01-02 00:00:00
Start: 2020-01-01T00:00:00
End: 2020-01-02T00:00:00
ID: 2020-01-01T00:00:00-2020-01-02T00:00:00

Cycle Equality and Comparison#

Cycles can be compared for equality using the == operator. The comparison behavior is designed to be flexible and intuitive for workflow operations.

Equality Rules:

Same type cycles: Two cycles are equal if they have the same begin_date and end_date
Mixed type cycles: A single date cycle equals an interval cycle if they share the same begin_date
String comparison: Cycles can be compared with ISO 8601 formatted strings

Example 1: Same Type Comparison

from woom.iters import Cycle

# Two single date cycles
cycle1 = Cycle("2020-01-01")
cycle2 = Cycle("2020-01-01")
assert cycle1 == cycle2  # True - same date

cycle3 = Cycle("2020-01-02")
assert cycle1 != cycle3  # True - different dates

# Two interval cycles
interval1 = Cycle("2020-01-01", "2020-01-10")
interval2 = Cycle("2020-01-01", "2020-01-10")
assert interval1 == interval2  # True - same interval

interval3 = Cycle("2020-01-01", "2020-01-15")
assert interval1 != interval3  # True - different end dates

Example 2: Mixed Type Comparison

# Single date vs interval - compared by begin_date
single = Cycle("2020-01-01")
interval = Cycle("2020-01-01", "2020-01-10")
assert single == interval  # True - same begin_date

# Different begin dates
single2 = Cycle("2020-01-02")
assert single2 != interval  # True - different begin_date

Example 3: String Comparison

cycle = Cycle("2020-01-01")
assert cycle == "2020-01-01T00:00:00+00:00"  # True

interval = Cycle("2020-01-01", "2020-01-10")
assert interval == "2020-01-01T00:00:00+00:00-2020-01-10T00:00:00+00:00"  # True

Use Cases:

This flexible equality behavior is useful for:

Task dependencies: Tasks can depend on cycles specified as dates or intervals interchangeably
Cycle lookups: Find cycles by date string without knowing if it’s stored as interval or point
Workflow logic: Simplify conditional logic when cycles might be configured differently

# In workflow code, these all work equivalently
task_cycle = workflow.get_task_cycle(task_name, "2020-01-01")

# Whether the workflow uses:
# - Single dates: Cycle("2020-01-01")
# - Intervals: Cycle("2020-01-01", "2020-01-02")
# Both match the string "2020-01-01"

Ensemble Member Variables#

When No Ensemble#

If no ensemble configured:

{{ member }}    # None

When Using Ensemble#

Basic Information:

{{ member.id }}       # Integer: 1, 2, 3, ...
{{ member.label }}    # String: "member001", "member002", ...

Custom Parameters:

From [[iters]] section:

[ensemble]
size = 3
    [[iters]]
    temperature = 10, 15, 20
    scenario = cold, medium, warm

Template:

Member: {{ member.id }}
Label: {{ member.label }}
Temperature: {{ member.temperature }}
Scenario: {{ member.scenario }}

Results for member 2:

Member: 2
Label: member002
Temperature: 15
Scenario: medium

Custom Parameters#

Global Parameters#

Defined in [params] section of workflow.cfg.

Flat parameters (use underscores):

[params]
timestep = 300
grid_nx = 100
grid_ny = 200

Template:

dt = {{ params.timestep }}
nx = {{ params.grid_nx }}
ny = {{ params.grid_ny }}

Nested parameters (use ConfigObj subsections):

[params]
timestep = 300

    [[paths]]
    data_dir = /data/ocean
    forcing_dir = /data/forcing

Template:

dt = {{ params.timestep }}
data: {{ params.paths.data_dir }}
forcing: {{ params.paths.forcing_dir }}

Host-Specific Parameters#

Override per host:

[params]
nprocs = 128

    [[hosts]]
        [[[laptop]]]
        nprocs = 4

        [[[supercomputer]]]
        nprocs = 2048

Template uses appropriate value:

mpirun -n {{ params.nprocs }} ./model

Task-Specific Parameters#

Override per task:

[params]
memory = 16GB

    [[tasks]]
        [[[run_model]]]
        memory = 128GB

        [[[postprocess]]]
        memory = 32GB

Environment Variables#

System Variables#

Standard environment variables:

{{ task_env.USER }}
{{ task_env.HOME }}
{{ task_env.HOSTNAME }}
{{ task_env.PWD }}

Workflow Variables#

Set in [env_vars] section:

[env_vars]
DATA_ROOT = /data/ocean
OMP_NUM_THREADS = 8

Access:

{{ task_env.DATA_ROOT }}
{{ task_env.OMP_NUM_THREADS }}

Special Woom Variables#

Automatically set by woom:

{{ task_env.WOOM_WORKFLOW_DIR }}
{{ task_env.WOOM_TASK_NAME }}
{{ task_env.WOOM_RUN_DIR }}
{{ task_env.WOOM_CYCLE }}
{{ task_env.WOOM_MEMBER }}

Host Variables#

Scratch Directory#

{{ scratch_dir }}    # From host configuration

Useful for dynamic paths:

run_dir = {{ scratch_dir }}/{{ task_path }}

Complete Context Example#

Given this configuration:

workflow.cfg:

[app]
name = ocean_model
conf = tropical
exp = test01

[cycles]
begin_date = 2020-01-01
freq = 1D
ncycles = 2
as_intervals = True

[ensemble]
size = 2
    [[iters]]
    perturbation = 0.01, 0.02

[params]
timestep = 600
    [[paths]]
    output_base = /scratch/output

Template:

# Configuration for {{ app.name }}/{{ app.conf }}/{{ app.exp }}
# Task: {{ task.name }}
# Cycle: {{ cycle.begin_date }} to {{ cycle.end_date }}
# Member: {{ member.label }}

[simulation]
start_time = "{{ cycle.begin_date_str }}"
end_time = "{{ cycle.end_date_str }}"
timestep = {{ params.timestep }}
perturbation = {{ member.perturbation }}

[output]
directory = "{{ params.paths.output_base }}/{{ task_path }}"
filename = "output_{{ cycle.token }}_{{ member.label }}.nc"

Result (task=run_model, cycle 1, member 2):

# Configuration for ocean_model/tropical/test01
# Task: run_model
# Cycle: 2020-01-01 00:00:00 to 2020-01-02 00:00:00
# Member: member002

[simulation]
start_time = "2020-01-01T00:00:00"
end_time = "2020-01-02T00:00:00"
timestep = 600
perturbation = 0.02

[output]
directory = "/scratch/output/ocean_model/tropical/test01/2020-01-01T00:00:00-2020-01-02T00:00:00/run_model/member002"
filename = "output_2020-01-01T00:00:00-2020-01-02T00:00:00_member002.nc"

Advanced Techniques#

Conditional Content#

Use Jinja2 conditionals:

{% if cycle %}
# Cycle-specific settings
start_date = "{{ cycle.begin_date_str }}"
{% else %}
# No cycle - use default
start_date = "2020-01-01T00:00:00"
{% endif %}

{% if member %}
# Ensemble member {{ member.id }}
seed = {{ 1000 + member.id }}
{% else %}
# Single run
seed = 1000
{% endif %}

Loops#

Loop over collections:

# All ensemble members
{% if member is not none %}
Members:
{% for m in workflow.members %}
- {{ m.label }}: {{ m.perturbation }}
{% endfor %}
{% endif %}

Filters#

Apply Jinja2 filters:

# Uppercase
Model: {{ app.name | upper }}

# Path join
File: {{ [run_dir, 'output.nc'] | join('/') }}

# Default value
Value: {{ params.optional | default('default_value') }}

Custom Filters#

# ext/jinja_filters.py
from woom.ext import JINJA_FILTERS

def format_date(date, fmt='%Y%m%d'):
    return date.strftime(fmt)

JINJA_FILTERS['format_date'] = format_date

Use in templates:

Date: {{ cycle.date | format_date('%Y-%m-%d') }}

Accessing Context Programmatically#

From Python#

# Get context for specific task/cycle/member
context = workflow.get_context(
    task_name='run_model',
    cycle='2020-01-01',
    member=1
)

# Access variables
print(f"Task: {context['task_name']}")
print(f"Cycle: {context['cycle'].date}")
print(f"Member: {context['member'].id}")
print(f"Parameter: {context['params']['timestep']}")

# Set as active context
workflow.set_context(task_name='run_model', cycle='2020-01-01')

# Now use workflow.context
task_name = workflow.context['task_name']

Context Scope#

Context is task-specific:

# Different contexts for different tasks
ctx1 = workflow.get_context(task_name='task1', cycle='2020-01-01')
ctx2 = workflow.get_context(task_name='task2', cycle='2020-01-01')

# Same cycle, different task info
assert ctx1['task_name'] == 'task1'
assert ctx2['task_name'] == 'task2'

Best Practices#

Use Descriptive Variable Names: {{ params.model.timestep }} is clearer than {{ params.dt }}
Provide Defaults: Use {{ var | default('fallback') }} for optional variables
Document Parameters: Comment what each parameter controls in your configuration
Organize Parameters: Use nested sections in [params] for logical grouping
Test Templates: Render templates independently to verify output
Handle Missing Cycles/Members: Check for None before accessing cycle/member attributes
Keep It Simple: Don’t overuse complex Jinja2 logic - consider Python preprocessing instead
Consistent Naming: Use consistent naming conventions across workflow, tasks, and parameters

Common Patterns#

Date-Based Output Paths#

output = {{ scratch_dir }}/{{ app.name }}/{{ cycle.date | strftime('%Y/%m/%d') }}/output.nc

Conditional Paths#

{% if member %}
output = results_{{ cycle.token }}_{{ member.label }}.nc
{% else %}
output = results_{{ cycle.token }}.nc
{% endif %}

Resource Scaling#

{% if params.resolution == 'high' %}
mpirun -n 512 ./model
{% elif params.resolution == 'medium' %}
mpirun -n 128 ./model
{% else %}
mpirun -n 32 ./model
{% endif %}

Troubleshooting#

Undefined Variable#

Error: UndefinedError: 'cycle' is undefined

Cause: Accessing variable that doesn’t exist in current context

Fix: Check if variable exists or provide default:

{% if cycle %}{{ cycle.date }}{% else %}No cycle{% endif %}
{{ cycle.date | default('N/A') }}

Wrong Variable Type#

Error: AttributeError: 'NoneType' object has no attribute 'date'

Cause: Variable is None

Fix: Check before accessing attributes:

{% if cycle is not none %}
Date: {{ cycle.date }}
{% endif %}

Template Not Rendering#

Cause: Syntax errors, missing variables

Debug: 1. Test template with simple content first 2. Add variables incrementally 3. Check Jinja2 syntax 4. Verify context has expected variables

Migration Guide#

This section helps migrate workflows from older Woom versions to the current version.

Parameters Isolation (v2025.10+)#

Change: User-defined workflow parameters are now isolated in the params sub-dictionary.

Old Syntax (before v2025.10):

# Direct access to parameters
dt = {{ timestep }}
nx = {{ grid_nx }}
data = {{ data_dir }}

New Syntax (v2025.10+):

# Parameters accessed via params
dt = {{ params.timestep }}
nx = {{ params.grid_nx }}
data = {{ params.data_dir }}

Why: This change prevents naming conflicts between user parameters and built-in context variables, making templates more predictable and maintainable.

Migration Steps:

Add params. prefix to all user-defined parameters in templates
Environment variables also change: WOOM_TIMESTEP becomes WOOM_PARAMS_TIMESTEP
Test all templates after migration

Task Variable Prefixes (v2025.10+)#

Change: Task-specific context variables now use the task_ prefix.

Old Syntax (before v2025.10):

Working directory: {{ run_dir }}
Submission directory: {{ submission_dir }}
Script path: {{ script_path }}
Environment: {{ env.USER }}
Context JSON: {{ context_json }}

New Syntax (v2025.10+):

Working directory: {{ task_run_dir }}
Submission directory: {{ task_submission_dir }}
Script path: {{ task_script_path }}
Environment: {{ task_env.USER }}
Context JSON: {{ task_context_json }}

Why: The task_ prefix makes it explicit that these variables are task-specific, improving template readability and avoiding ambiguity with workflow-level variables.

Migration Steps:

Replace {{ run_dir }} with {{ task_run_dir }}
Replace {{ submission_dir }} with {{ task_submission_dir }}
Replace {{ script_path }} with {{ task_script_path }}
Replace {{ env. with {{ task_env.
Replace {{ context_json }} with {{ task_context_json }}

Backward Compatibility: For a limited time, the old names remain available as aliases to the new names, but they will be removed in a future version. Migrate as soon as possible.

Complete Migration Example#

Old template (before v2025.10):

#!/bin/bash
# Model configuration
cd {{ run_dir }}

export MODEL_TIMESTEP={{ timestep }}
export MODEL_GRIDSIZE={{ grid_nx }}x{{ grid_ny }}
export DATA_DIR={{ data_dir }}
export USER={{ env.USER }}

echo "Running in: $(pwd)"
echo "Job script: {{ script_path }}"
./model --config config.nml

New template (v2025.10+):

#!/bin/bash
# Model configuration
cd {{ task_run_dir }}

export MODEL_TIMESTEP={{ params.timestep }}
export MODEL_GRIDSIZE={{ params.grid_nx }}x{{ params.grid_ny }}
export DATA_DIR={{ params.data_dir }}
export USER={{ task_env.USER }}

echo "Running in: $(pwd)"
echo "Job script: {{ task_script_path }}"
./model --config config.nml

Context and Variables In-Depth

Contents

Context and Variables In-Depth#

What is the Context?#

Access in Templates#

Access in Python#

Workflow Variables#

Application Information#

Workflow Paths#

Task Variables#

Task Information#

Task Object#

Cycle Variables#

When No Cycles#

When Using Cycles#

Cycle Equality and Comparison#

Ensemble Member Variables#

When No Ensemble#

When Using Ensemble#

Custom Parameters#

Global Parameters#

Host-Specific Parameters#

Task-Specific Parameters#

Environment Variables#

System Variables#

Workflow Variables#

Special Woom Variables#

Host Variables#

Scratch Directory#

Complete Context Example#

Advanced Techniques#

Conditional Content#

Loops#

Filters#

Custom Filters#

Accessing Context Programmatically#

From Python#

Context Scope#

Best Practices#

Common Patterns#

Date-Based Output Paths#

Conditional Paths#

Resource Scaling#

Troubleshooting#

Undefined Variable#

Wrong Variable Type#

Template Not Rendering#

Migration Guide#

Parameters Isolation (v2025.10+)#

Task Variable Prefixes (v2025.10+)#

Complete Migration Example#

See Also#