Working with artifacts

About

This example demonstrates woom’s artifact system for tracking, validating, and managing workflow output files.

Artifacts provide a structured way to handle files that tasks produce and consume, ensuring data lineage and enabling automatic validation. This example shows three common artifact patterns:

1. Download artifact (download_clim task):

• Prolog task that downloads a climatology file

• Registers the file as an artifact named clim_file

• Makes the artifact available to downstream tasks throughout the workflow

• Demonstrates artifact generation in setup stages

2. Dynamic artifact generation (run_model task):

• Cyclic task that processes data and generates daily output files

• Uses the gen_daily_files custom function extension to create multiple artifacts per cycle

• Accesses the climatology artifact from the prolog stage

• Copies the last daily file from the previous cycle (prev_croco_rst.nc) for restart capability

• Shows inter-cycle artifact dependencies

3. Artifact aggregation (concat_nc task):

• Epilog task that consolidates all daily artifacts from run_model

• Merges multiple cycle outputs into a single merged artifact

• Demonstrates post-processing patterns for time series data

This example is essential for workflows that need to track data provenance, validate outputs, share files between tasks, or implement restart capabilities.

Path: examples/academic/artifacts.

Configuring

workflow.cfg

[app]
name=academic_artifacts

[cycles]
begin_date=2020-01-01
end_date=2020-01-02
freq=1D
as_intervals=False

[stages]
    [[prolog]]
    downloads=download_clim
    [[cycles]]
    run=run_croco
    [[epilog]]
    finalize=concat_nc

tasks.cfg

[download_clim]

    [[content]]
    commandline=echo foo > {{ task.artifacts['clim_file'] }} || exit 1

    [[artifacts]]
        [[[clim_file]]]
        path={{ task_run_dir }}/clim.nc

[run_croco]

    [[content]]
    commandline="""
# - get the climatoogy
cp {{ workflow.get_task_artifact_paths('clim_file', 'download_clim') }} . || exit 1

{% if not cycle.is_first %}
# - get the restart from previous cycle
echo {{ task.name }} {{ cycle.prev }}
cp {{ workflow.get_task_artifact_paths('model_restart', task.name, cycle.prev)[-1] }} prev_croco_rst.nc || exit 1
{% endif %}

# - run the model
{% for path in task.artifacts['model_restart'] -%}
echo {{ task.name }} {{ cycle }} > {{ path }} || exit 1
{% endfor %}
"""

    [[artifacts]]
        [[[model_restart]]]
        path=gen_hourly_files
        callable=True
            [[[[kwargs]]]]
            path_format={task_run_dir}/croco_{time:%Y-%m-%d-%H-%M}.nc


[concat_nc]

    [[content]]
    commandline="echo concat {{ workflow.get_task_artifact_paths('model_restart', 'run_croco', flat=True)|join(' ') }} > {{ task.artifacts['merged'] }} || exit 1"

    [[artifacts]]
        [[[merged]]]
        path={{ task_run_dir }}/merged.nc

Extending

ext/artifacts_generators.py

import pandas as pd


def gen_hourly_files(**kwargs):
    date0 = kwargs["cycle"].date  # provided by the context
    date1 = date0.add(kwargs["cycles_freq"])
    path_format = kwargs["path_format"]  # provided by the tasks.cfg file
    hours = pd.date_range(date0, date1, freq="1h", inclusive="left")
    paths = []
    for time in hours:
        paths.append(path_format.format(time=time, **kwargs))
    return paths


ARTIFACTS_GENERATORS = {"gen_hourly_files": gen_hourly_files}

Running

Overview

Let’s have an overview of stages before running the workflow.

$ woom show overview
##################################### APP ######################################
name: academic_artifacts
################################## TASK TREE ###################################
prolog:
    - downloads: download_clim
cycles:
    - run: run_croco
epilog:
    - finalize: concat_nc
#################################### CYCLES ####################################
2020-01-01T00:00:00+00:00
2020-01-02T00:00:00+00:00
################################### ENSEMBLE ###################################
no member

Dry run

Now let’s run the workflow in test (dry) and debug modes.

$ woom run --log-no-color --log-level debug --dry-run
woom.log    : DEBUG    *** STARTED LOG SESSION ***
woom.cli    : DEBUG    Run the workflow
woom.cli    : DEBUG    Loading extensions
woom.cli    : INFO     Loaded extensions: artifacts_generators
woom.cli    : DEBUG    Load workflow config: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/workflow.cfg
woom.cli    : INFO     Loaded workflow config
woom.cli    : INFO     App name: academic_artifacts
woom.cli    : DEBUG    Initialize the host manager
woom.cli    : INFO     Initialized the host manager
woom.cli    : DEBUG    Load hosts config file: hosts.cfg
woom.cli    : INFO     Loaded hosts config file: hosts.cfg
woom.cli    : DEBUG    Infer host
woom.cli    : INFO     Infered host: local
woom.cli    : DEBUG    Initialize the task manager
woom.cli    : INFO     Initialized the task manager
woom.cli    : DEBUG    Load the task config file: tasks.cfg
woom.cli    : INFO     Loaded the task config file: tasks.cfg
woom.cli    : DEBUG    Initialize the workflow
woom.workflow: DEBUG    Task tree:
prolog:
    - downloads: download_clim
cycles:
    - run: run_croco
epilog:
    - finalize: concat_nc
woom.cli    : INFO     Initialized the workflow
woom.cli    : INFO     Successfully setup the workflow!
woom.cli    : DEBUG    Run the workflow
woom.workflow: DEBUG    Running the workflow in fake mode
woom.workflow: DEBUG    Entering stage: prolog
woom.workflow: DEBUG    Entering sequence: downloads
woom.workflow: DEBUG    Running task: prolog/downloads/download_clim
woom.workflow: DEBUG    Task path: academic_artifacts/prolog/download_clim
woom.workflow: DEBUG    Cleaning task: prolog/downloads/download_clim
woom.job    : INFO     Started job manager: BackgroundJobManager()
woom.workflow: DEBUG    Submitting task: prolog/downloads/download_clim
woom.workflow: DEBUG      Dependencies: 
woom.workflow: DEBUG    Creating directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim
woom.workflow: INFO     Fake submission: academic_artifacts/prolog/download_clim → bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.sh
woom.workflow: DEBUG    Fake submission:
--------------- submission command ---------------
bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.sh
-------------- batch script content --------------
#!/bin/bash

# Prolog
set -eo pipefail
# - handler for graceful termination
on_sigterm() {
    echo "Received termination signal, cleaning up..."
    # Just exit cleanly, let on_exit handle status
    exit 0
}
# - handler for killing termination
on_sigkill() {
    echo "Received kill signal, cleaning up..." >&2
    # Just exit cleanly, let on_exit handle status
    exit 1
}
# - handler for exit (always called)
on_exit() {
    status=$?
    if [ ! -f "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.terminating" ]; then
        echo $status > "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.status"
    fi
    exit $status
}
trap on_sigterm SIGKILL
trap on_sigkill SIGTERM SIGINT
trap on_exit EXIT


# Environment variables
export WOOM_TASK_NAME="download_clim"
export WOOM_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim"
export WOOM_WORKFLOW="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/workflow.cfg"
export WOOM_HOST="local"
export WOOM_TASKMANAGER="/home/docs/checkouts/readthedocs.org/user_builds/woom/envs/latest/lib/python3.11/site-packages/woom/tasks.cfg:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/tasks.cfg"
export WOOM_JOBMANAGER="BackgroundJobManager"
export WOOM_TASK_TREE_PROLOG_DOWNLOADS="['download_clim']"
export WOOM_TASK_TREE_CYCLES_RUN="['run_croco']"
export WOOM_TASK_TREE_EPILOG_FINALIZE="['concat_nc']"
export WOOM_CYCLES="2020-01-01T00:00:00+00:00:2020-01-02T00:00:00+00:00"
export WOOM_NMEMBERS="0"
export WOOM_MEMBERS=""
export WOOM_PATHS_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin"
export WOOM_PATHS_PYTHONPATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python"
export WOOM_PATHS_LIBRARY_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib"
export WOOM_PATHS_INCLUDE_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include"
export WOOM_APP_PATH="academic_artifacts"
export WOOM_APP_NAME="academic_artifacts"
export WOOM_APP_CONF=""
export WOOM_APP_EXP=""
export WOOM_CYCLES_BEGIN_DATE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLES_END_DATE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLES_FREQ="1D"
export WOOM_CYCLES_AS_INTERVALS="0"
export WOOM_CYCLES_ROUND=""
export WOOM_CYCLES_NCYCLES="0"
export WOOM_CYCLES_INDEP="0"
export WOOM_CYCLES_HORIZON=""
export WOOM_SCRATCH_DIR="/home/docs/woom/scratch"
export WOOM_WORK_DIR="/home/docs/woom/work"
export WOOM_WORKFLOW_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts"
export WOOM_LOG_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/log"
export WOOM_CYCLE="prolog"
export WOOM_MEMBER=""
export WOOM_TASK="download_clim"
export WOOM_TASK_PATH="academic_artifacts/prolog/download_clim"
export WOOM_TASK_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim"
export WOOM_TASK_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim"
export WOOM_TASK_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.sh"
export WOOM_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim"
export WOOM_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.sh"
export WOOM_TASK_ENV=""
export WOOM_ENV=""
export WOOM_TASK_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/context.json"
export WOOM_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/context.json"
export PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin:$PATH
export PYTHONPATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python:$PYTHONPATH
export LIBRARY_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib:$LIBRARY_PATH
export INCLUDE_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include:$INCLUDE_PATH


# Go to run dir
{ mkdir -p /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim; cd /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim; } || exit 1

# Run the commandline(s)
echo foo > /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc || exit 1

# Check artifacts
test -f "/home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc" || { echo artifact not created: clim_file=/home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc; exit 1; }
  
------------------- artifacts --------------------
clim_file: /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc
--------------------------------------------------
woom.workflow: INFO     Submitted task: prolog/downloads/download_clim with job id 931548
woom.workflow: INFO     Successfully submitted stage: prolog
woom.workflow: DEBUG    Entering stage: cycles
woom.workflow: INFO     Cycling on dates from 2020-01-01 00:00:00+00:00 to 2020-01-02 00:00:00+00:00 in 2 time(s)
woom.workflow: DEBUG    Running cycle: 2020-01-01T00:00:00+00:00
woom.workflow: DEBUG    Entering sequence: run
woom.workflow: DEBUG    Running task: cycles/run/run_croco
woom.workflow: DEBUG    Task path: academic_artifacts/2020-01-01T00:00:00+00:00/run_croco
woom.workflow: DEBUG    Cleaning task: cycles/run/run_croco
woom.workflow: DEBUG    Submitting task: cycles/run/run_croco
woom.workflow: DEBUG      Dependencies: 931548
woom.workflow: DEBUG    Creating directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco
woom.workflow: INFO     Fake submission: academic_artifacts/2020-01-01T00:00:00+00:00/run_croco → bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.sh
woom.workflow: DEBUG    Fake submission:
--------------- submission command ---------------
bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.sh
-------------- batch script content --------------
#!/bin/bash

# Prolog
set -eo pipefail
# - handler for graceful termination
on_sigterm() {
    echo "Received termination signal, cleaning up..."
    # Just exit cleanly, let on_exit handle status
    exit 0
}
# - handler for killing termination
on_sigkill() {
    echo "Received kill signal, cleaning up..." >&2
    # Just exit cleanly, let on_exit handle status
    exit 1
}
# - handler for exit (always called)
on_exit() {
    status=$?
    if [ ! -f "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.terminating" ]; then
        echo $status > "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.status"
    fi
    exit $status
}
trap on_sigterm SIGKILL
trap on_sigkill SIGTERM SIGINT
trap on_exit EXIT


# Environment variables
export WOOM_TASK_NAME="run_croco"
export WOOM_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco"
export WOOM_WORKFLOW="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/workflow.cfg"
export WOOM_HOST="local"
export WOOM_TASKMANAGER="/home/docs/checkouts/readthedocs.org/user_builds/woom/envs/latest/lib/python3.11/site-packages/woom/tasks.cfg:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/tasks.cfg"
export WOOM_JOBMANAGER="BackgroundJobManager"
export WOOM_TASK_TREE_PROLOG_DOWNLOADS="['download_clim']"
export WOOM_TASK_TREE_CYCLES_RUN="['run_croco']"
export WOOM_TASK_TREE_EPILOG_FINALIZE="['concat_nc']"
export WOOM_CYCLES="2020-01-01T00:00:00+00:00:2020-01-02T00:00:00+00:00"
export WOOM_NMEMBERS="0"
export WOOM_MEMBERS=""
export WOOM_PATHS_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin"
export WOOM_PATHS_PYTHONPATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python"
export WOOM_PATHS_LIBRARY_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib"
export WOOM_PATHS_INCLUDE_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include"
export WOOM_APP_PATH="academic_artifacts"
export WOOM_APP_NAME="academic_artifacts"
export WOOM_APP_CONF=""
export WOOM_APP_EXP=""
export WOOM_CYCLES_BEGIN_DATE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLES_END_DATE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLES_FREQ="1D"
export WOOM_CYCLES_AS_INTERVALS="0"
export WOOM_CYCLES_ROUND=""
export WOOM_CYCLES_NCYCLES="0"
export WOOM_CYCLES_INDEP="0"
export WOOM_CYCLES_HORIZON=""
export WOOM_SCRATCH_DIR="/home/docs/woom/scratch"
export WOOM_WORK_DIR="/home/docs/woom/work"
export WOOM_WORKFLOW_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts"
export WOOM_LOG_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/log"
export WOOM_CYCLE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_BEGIN_DATE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_LABEL="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_TOKEN="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_DATE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_IS_FIRST="1"
export WOOM_CYCLE_IS_LAST="0"
export WOOM_CYCLE_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_PREV=""
export WOOM_CYCLE_BEGIN_DATE_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_LABEL_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_TOKEN_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_DATE_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_IS_FIRST_NEXT="0"
export WOOM_CYCLE_IS_LAST_NEXT="1"
export WOOM_CYCLE_NEXT_NEXT=""
export WOOM_CYCLE_PREV_NEXT="2020-01-01T00:00:00+00:00"
export WOOM_MEMBER=""
export WOOM_TASK="run_croco"
export WOOM_TASK_PATH="academic_artifacts/2020-01-01T00:00:00+00:00/run_croco"
export WOOM_TASK_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco"
export WOOM_TASK_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco"
export WOOM_TASK_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.sh"
export WOOM_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco"
export WOOM_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.sh"
export WOOM_TASK_ENV=""
export WOOM_ENV=""
export WOOM_TASK_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/context.json"
export WOOM_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/context.json"
export PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin:$PATH
export PYTHONPATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python:$PYTHONPATH
export LIBRARY_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib:$LIBRARY_PATH
export INCLUDE_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include:$INCLUDE_PATH


# Go to run dir
{ mkdir -p /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco; cd /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco; } || exit 1

# Run the commandline(s)

# - get the climatoogy
cp /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc . || exit 1


# - run the model
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc || exit 1
echo run_croco 2020-01-01T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc || exit 1


# Check artifacts
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc; exit 1; }
      
------------------- artifacts --------------------
model_restart: /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc
--------------------------------------------------
woom.workflow: INFO     Submitted task: cycles/run/run_croco with job id 530187 depending on [931548]
woom.workflow: INFO     Successfully submitted cycle: 2020-01-01T00:00:00+00:00
woom.workflow: DEBUG    Running cycle: 2020-01-02T00:00:00+00:00
woom.workflow: DEBUG    Entering sequence: run
woom.workflow: DEBUG    Running task: cycles/run/run_croco
woom.workflow: DEBUG    Task path: academic_artifacts/2020-01-02T00:00:00+00:00/run_croco
woom.workflow: DEBUG    Cleaning task: cycles/run/run_croco
woom.workflow: DEBUG    Submitting task: cycles/run/run_croco
woom.workflow: DEBUG      Dependencies: 530187
woom.workflow: DEBUG    Creating directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco
woom.workflow: INFO     Fake submission: academic_artifacts/2020-01-02T00:00:00+00:00/run_croco → bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.sh
woom.workflow: DEBUG    Fake submission:
--------------- submission command ---------------
bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.sh
-------------- batch script content --------------
#!/bin/bash

# Prolog
set -eo pipefail
# - handler for graceful termination
on_sigterm() {
    echo "Received termination signal, cleaning up..."
    # Just exit cleanly, let on_exit handle status
    exit 0
}
# - handler for killing termination
on_sigkill() {
    echo "Received kill signal, cleaning up..." >&2
    # Just exit cleanly, let on_exit handle status
    exit 1
}
# - handler for exit (always called)
on_exit() {
    status=$?
    if [ ! -f "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.terminating" ]; then
        echo $status > "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.status"
    fi
    exit $status
}
trap on_sigterm SIGKILL
trap on_sigkill SIGTERM SIGINT
trap on_exit EXIT


# Environment variables
export WOOM_TASK_NAME="run_croco"
export WOOM_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco"
export WOOM_WORKFLOW="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/workflow.cfg"
export WOOM_HOST="local"
export WOOM_TASKMANAGER="/home/docs/checkouts/readthedocs.org/user_builds/woom/envs/latest/lib/python3.11/site-packages/woom/tasks.cfg:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/tasks.cfg"
export WOOM_JOBMANAGER="BackgroundJobManager"
export WOOM_TASK_TREE_PROLOG_DOWNLOADS="['download_clim']"
export WOOM_TASK_TREE_CYCLES_RUN="['run_croco']"
export WOOM_TASK_TREE_EPILOG_FINALIZE="['concat_nc']"
export WOOM_CYCLES="2020-01-01T00:00:00+00:00:2020-01-02T00:00:00+00:00"
export WOOM_NMEMBERS="0"
export WOOM_MEMBERS=""
export WOOM_PATHS_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin"
export WOOM_PATHS_PYTHONPATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python"
export WOOM_PATHS_LIBRARY_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib"
export WOOM_PATHS_INCLUDE_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include"
export WOOM_APP_PATH="academic_artifacts"
export WOOM_APP_NAME="academic_artifacts"
export WOOM_APP_CONF=""
export WOOM_APP_EXP=""
export WOOM_CYCLES_BEGIN_DATE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLES_END_DATE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLES_FREQ="1D"
export WOOM_CYCLES_AS_INTERVALS="0"
export WOOM_CYCLES_ROUND=""
export WOOM_CYCLES_NCYCLES="0"
export WOOM_CYCLES_INDEP="0"
export WOOM_CYCLES_HORIZON=""
export WOOM_SCRATCH_DIR="/home/docs/woom/scratch"
export WOOM_WORK_DIR="/home/docs/woom/work"
export WOOM_WORKFLOW_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts"
export WOOM_LOG_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/log"
export WOOM_CYCLE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_BEGIN_DATE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_LABEL="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_TOKEN="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_DATE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_IS_FIRST="0"
export WOOM_CYCLE_IS_LAST="1"
export WOOM_CYCLE_NEXT=""
export WOOM_CYCLE_PREV="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_BEGIN_DATE_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_LABEL_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_TOKEN_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_DATE_NEXT="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_IS_FIRST_NEXT="0"
export WOOM_CYCLE_IS_LAST_NEXT="1"
export WOOM_CYCLE_NEXT_NEXT=""
export WOOM_CYCLE_PREV_NEXT="2020-01-01T00:00:00+00:00"
export WOOM_MEMBER=""
export WOOM_TASK="run_croco"
export WOOM_TASK_PATH="academic_artifacts/2020-01-02T00:00:00+00:00/run_croco"
export WOOM_TASK_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco"
export WOOM_TASK_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco"
export WOOM_TASK_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.sh"
export WOOM_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco"
export WOOM_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.sh"
export WOOM_TASK_ENV=""
export WOOM_ENV=""
export WOOM_TASK_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/context.json"
export WOOM_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/context.json"
export WOOM_CYCLE_BEGIN_DATE_PREV="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_LABEL_PREV="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_TOKEN_PREV="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_DATE_PREV="2020-01-01T00:00:00+00:00"
export WOOM_CYCLE_IS_FIRST_PREV="1"
export WOOM_CYCLE_IS_LAST_PREV="0"
export WOOM_CYCLE_NEXT_PREV="2020-01-02T00:00:00+00:00"
export WOOM_CYCLE_PREV_PREV=""
export PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin:$PATH
export PYTHONPATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python:$PYTHONPATH
export LIBRARY_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib:$LIBRARY_PATH
export INCLUDE_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include:$INCLUDE_PATH


# Go to run dir
{ mkdir -p /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco; cd /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco; } || exit 1

# Run the commandline(s)

# - get the climatoogy
cp /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc . || exit 1

# - get the restart from previous cycle
echo run_croco 2020-01-01T00:00:00+00:00
cp /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc prev_croco_rst.nc || exit 1

# - run the model
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc || exit 1
echo run_croco 2020-01-02T00:00:00+00:00 > /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc || exit 1


# Check artifacts
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc; exit 1; }
    test -f "/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc" || { echo artifact not created: model_restart=/home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc; exit 1; }
      
------------------- artifacts --------------------
model_restart: /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc, /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc
--------------------------------------------------
woom.workflow: INFO     Submitted task: cycles/run/run_croco with job id 866319 depending on [530187]
woom.workflow: INFO     Successfully submitted cycle: 2020-01-02T00:00:00+00:00
woom.workflow: DEBUG    Entering stage: epilog
woom.workflow: DEBUG    Entering sequence: finalize
woom.workflow: DEBUG    Running task: epilog/finalize/concat_nc
woom.workflow: DEBUG    Task path: academic_artifacts/epilog/concat_nc
woom.workflow: DEBUG    Cleaning task: epilog/finalize/concat_nc
woom.workflow: DEBUG    Submitting task: epilog/finalize/concat_nc
woom.workflow: DEBUG      Dependencies: 866319
woom.workflow: DEBUG    Creating directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc
woom.workflow: INFO     Fake submission: academic_artifacts/epilog/concat_nc → bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.sh
woom.workflow: DEBUG    Fake submission:
--------------- submission command ---------------
bash /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.sh
-------------- batch script content --------------
#!/bin/bash

# Prolog
set -eo pipefail
# - handler for graceful termination
on_sigterm() {
    echo "Received termination signal, cleaning up..."
    # Just exit cleanly, let on_exit handle status
    exit 0
}
# - handler for killing termination
on_sigkill() {
    echo "Received kill signal, cleaning up..." >&2
    # Just exit cleanly, let on_exit handle status
    exit 1
}
# - handler for exit (always called)
on_exit() {
    status=$?
    if [ ! -f "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.terminating" ]; then
        echo $status > "/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.status"
    fi
    exit $status
}
trap on_sigterm SIGKILL
trap on_sigkill SIGTERM SIGINT
trap on_exit EXIT


# Environment variables
export WOOM_TASK_NAME="concat_nc"
export WOOM_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc"
export WOOM_WORKFLOW="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/workflow.cfg"
export WOOM_HOST="local"
export WOOM_TASKMANAGER="/home/docs/checkouts/readthedocs.org/user_builds/woom/envs/latest/lib/python3.11/site-packages/woom/tasks.cfg:/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/tasks.cfg"
export WOOM_JOBMANAGER="BackgroundJobManager"
export WOOM_TASK_TREE_PROLOG_DOWNLOADS="['download_clim']"
export WOOM_TASK_TREE_CYCLES_RUN="['run_croco']"
export WOOM_TASK_TREE_EPILOG_FINALIZE="['concat_nc']"
export WOOM_CYCLES="2020-01-01T00:00:00+00:00:2020-01-02T00:00:00+00:00"
export WOOM_NMEMBERS="0"
export WOOM_MEMBERS=""
export WOOM_PATHS_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin"
export WOOM_PATHS_PYTHONPATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python"
export WOOM_PATHS_LIBRARY_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib"
export WOOM_PATHS_INCLUDE_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include"
export WOOM_APP_PATH="academic_artifacts"
export WOOM_APP_NAME="academic_artifacts"
export WOOM_APP_CONF=""
export WOOM_APP_EXP=""
export WOOM_CYCLES_BEGIN_DATE="2020-01-01T00:00:00+00:00"
export WOOM_CYCLES_END_DATE="2020-01-02T00:00:00+00:00"
export WOOM_CYCLES_FREQ="1D"
export WOOM_CYCLES_AS_INTERVALS="0"
export WOOM_CYCLES_ROUND=""
export WOOM_CYCLES_NCYCLES="0"
export WOOM_CYCLES_INDEP="0"
export WOOM_CYCLES_HORIZON=""
export WOOM_SCRATCH_DIR="/home/docs/woom/scratch"
export WOOM_WORK_DIR="/home/docs/woom/work"
export WOOM_WORKFLOW_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts"
export WOOM_LOG_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/log"
export WOOM_CYCLE="epilog"
export WOOM_MEMBER=""
export WOOM_TASK="concat_nc"
export WOOM_TASK_PATH="academic_artifacts/epilog/concat_nc"
export WOOM_TASK_RUN_DIR="/home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc"
export WOOM_TASK_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc"
export WOOM_TASK_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.sh"
export WOOM_SUBMISSION_DIR="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc"
export WOOM_SCRIPT_PATH="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.sh"
export WOOM_TASK_ENV=""
export WOOM_ENV=""
export WOOM_TASK_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/context.json"
export WOOM_CONTEXT_JSON="/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/context.json"
export PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/bin:$PATH
export PYTHONPATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib/python:$PYTHONPATH
export LIBRARY_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/lib:$LIBRARY_PATH
export INCLUDE_PATH=/home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/include:$INCLUDE_PATH


# Go to run dir
{ mkdir -p /home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc; cd /home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc; } || exit 1

# Run the commandline(s)
echo concat /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc > /home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc/merged.nc || exit 1

# Check artifacts
test -f "/home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc/merged.nc" || { echo artifact not created: merged=/home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc/merged.nc; exit 1; }
  
------------------- artifacts --------------------
merged: /home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc/merged.nc
--------------------------------------------------
woom.workflow: INFO     Submitted task: epilog/finalize/concat_nc with job id 529285 depending on [866319]
woom.workflow: INFO     Successfully submitted stage: epilog
woom.workflow: DEBUG    Terminating non-blocking jobs
woom.cli    : INFO     Successfully ran the workflow!

Normal run

And finally in run it.

$ woom run --log-no-color
woom.cli    : INFO     Loaded extensions: artifacts_generators
woom.cli    : INFO     Loaded workflow config
woom.cli    : INFO     App name: academic_artifacts
woom.cli    : INFO     Initialized the host manager
woom.cli    : INFO     Loaded hosts config file: hosts.cfg
woom.cli    : INFO     Infered host: local
woom.cli    : INFO     Initialized the task manager
woom.cli    : INFO     Loaded the task config file: tasks.cfg
woom.cli    : INFO     Initialized the workflow
woom.cli    : INFO     Successfully setup the workflow!
woom.job    : INFO     Started job manager: BackgroundJobManager()
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim
woom.workflow: INFO     Created batch script: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/prolog/download_clim/job.sh
woom.workflow: INFO     Submitted task: prolog/downloads/download_clim with job id 817
woom.workflow: INFO     Successfully submitted stage: prolog
woom.workflow: INFO     Cycling on dates from 2020-01-01 00:00:00+00:00 to 2020-01-02 00:00:00+00:00 in 2 time(s)
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco
woom.workflow: INFO     Created batch script: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/job.sh
woom.workflow: INFO     Submitted task: cycles/run/run_croco with job id 819 depending on [817]
woom.workflow: INFO     Successfully submitted cycle: 2020-01-01T00:00:00+00:00
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco
woom.workflow: INFO     Created batch script: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/job.sh
woom.workflow: INFO     Submitted task: cycles/run/run_croco with job id 822 depending on [819]
woom.workflow: INFO     Successfully submitted cycle: 2020-01-02T00:00:00+00:00
woom.workflow: INFO     Created directory: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc
woom.workflow: INFO     Created batch script: /home/docs/checkouts/readthedocs.org/user_builds/woom/checkouts/latest/examples/academic/artifacts/jobs/academic_artifacts/epilog/concat_nc/job.sh
woom.workflow: INFO     Submitted task: epilog/finalize/concat_nc with job id 826 depending on [822]
woom.workflow: INFO     Successfully submitted stage: epilog
woom.cli    : INFO     Successfully ran the workflow!

Check status

Check what is running or finished.

$ woom show status
╭──────────┬─────────┬───────────────┬───────────────────────────┬─────────────────────────────────────────────────────────────╮
│ STATUS   │   JOBID │ TASK          │ CYCLE                     │ SUBMISSION DIR                                              │
├──────────┼─────────┼───────────────┼───────────────────────────┼─────────────────────────────────────────────────────────────┤
│ SUCCESS  │     817 │ download_clim │ prolog                    │ jobs/academic_artifacts/prolog/download_clim                │
│ SUCCESS  │     819 │ run_croco     │ 2020-01-01T00:00:00+00:00 │ jobs/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco │
│ SUCCESS  │     822 │ run_croco     │ 2020-01-02T00:00:00+00:00 │ jobs/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco │
│ SUCCESS  │     826 │ concat_nc     │ epilog                    │ jobs/academic_artifacts/epilog/concat_nc                    │
╰──────────┴─────────┴───────────────┴───────────────────────────┴─────────────────────────────────────────────────────────────╯

Show run directories

Show where tasks were executed.

$ woom show run_dirs
╭───────────────┬───────────────────────────┬─────────────────────────────────────────────────────────────────────────────────────╮
│ TASK          │ CYCLE                     │ RUN DIR                                                                             │
├───────────────┼───────────────────────────┼─────────────────────────────────────────────────────────────────────────────────────┤
│ download_clim │ prolog                    │ /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim                │
│ run_croco     │ 2020-01-01T00:00:00+00:00 │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco │
│ run_croco     │ 2020-01-02T00:00:00+00:00 │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco │
│ concat_nc     │ epilog                    │ /home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc                    │
╰───────────────┴───────────────────────────┴─────────────────────────────────────────────────────────────────────────────────────╯

Show artifacts

Show artifacts, either generated or expected.

$ woom show artifacts
╭───────────────┬───────────────┬───────────────────────────────────────────────────────────────────────────────────────────────────────────────┬───────────╮
│ TASK          │ ARTIFACT      │ PATH                                                                                                          │ EXISTS?   │
├───────────────┼───────────────┼───────────────────────────────────────────────────────────────────────────────────────────────────────────────┼───────────┤
│ download_clim │ clim_file     │ /home/docs/woom/scratch/woom/academic_artifacts/prolog/download_clim/clim.nc                                  │ False     │
│ run_croco     │ model_restart │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-01T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc │ False     │
│ run_croco     │ model_restart │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-00-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-01-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-02-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-03-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-04-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-05-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-06-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-07-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-08-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-09-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-10-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-11-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-12-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-13-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-14-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-15-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-16-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-17-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-18-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-19-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-20-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-21-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-22-00.nc │ False     │
│ run_croco     │               │ /home/docs/woom/scratch/woom/academic_artifacts/2020-01-02T00:00:00+00:00/run_croco/croco_2020-01-01-23-00.nc │ False     │
│ concat_nc     │ merged        │ /home/docs/woom/scratch/woom/academic_artifacts/epilog/concat_nc/merged.nc                                    │ False     │
╰───────────────┴───────────────┴───────────────────────────────────────────────────────────────────────────────────────────────────────────────┴───────────╯