Usage¶

This document provides a quick guide on all commands of wrf-ensembly. They are written here mostly in the order they are used in a typical workflow, so this could also be used as a quick start guide. Commands are grouped by their functionality in the following sections:

Experiment Management
Preprocessing
Observations
Ensemble Management
Postprocessing
Status
SLURM

For a new experiment, you will typically start with creating it and copying the model (experiment management), then preprocess the input data (preprocessing), preprocess observations (observations), run the ensemble (ensemble management), and finally postprocess the results (postprocessing). You can check the experiment status at any time using the status commands. If you are using SLURM, you can also find commands for that in the last section (preprocess, run ensemble, postprocess).

All commands will take the path to the experiment directory as the first argument. This directory will contain the model data, input and output forecasts, configuration and anything else related to the experiment. It must be writable by the current user.

Experiment Management¶

wrf-ensembly EXPERIMENT_PATH experiment create¶

Create a new experiment directory at EXPERIMENT_PATH. The TEMPLATE argument specifies which config file template to use when creating the experiment. Available templates are located in wrf_ensembly/config_templates/. The template name should not include the .toml extension, e.g., default for default.toml.

If in doubt, use the iridium_chem_4.6.0 template and modify it to your needs.

Usage:

wrf-ensembly EXPERIMENT_PATH experiment create [OPTIONS] TEMPLATE

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH experiment copy-model¶

Setup the experiment by doing the following

Usage:

wrf-ensembly EXPERIMENT_PATH experiment copy-model [OPTIONS]

Options:

  --force  If model directory already exists, remove and copy again
  --help   Show this message and exit.

wrf-ensembly EXPERIMENT_PATH experiment cycle-info¶

Prints cycle information

Usage:

wrf-ensembly EXPERIMENT_PATH experiment cycle-info [OPTIONS]

Options:

  --to-csv PATH  Write cycle info to CSV file
  --help         Show this message and exit.

Preprocessing¶

wrf-ensembly EXPERIMENT_PATH preprocess setup¶

Setups the preprocessing environment by copying WRF/WPS to the correct places and generating their namelists.

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess setup [OPTIONS]

Options:

  --only-namelist  If set, do not copy WPS/WRF, only generate namelist
  --help           Show this message and exit.

wrf-ensembly EXPERIMENT_PATH preprocess geogrid¶

Runs geogrid.exe for the experiment.

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess geogrid [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH preprocess ungrib¶

Runs ungrib.exe for the experiment, after linking the grib files into the WPS directory

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess ungrib [OPTIONS]

Options:

  --member INTEGER  When using different IC/BC for each member, which member
                    to ungrib
  --help            Show this message and exit.

wrf-ensembly EXPERIMENT_PATH preprocess metgrid¶

Run metgrid.exe to produce the met_em*.nc files.

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess metgrid [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH preprocess real¶

Run real.exe to produce the initial (wrfinput) and boundary (wrfbdy) conditions the given CYCLE. You should run this for all cycles to have initial/boundary conditions for your experiment.

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess real [OPTIONS]

Options:

  --cycle INTEGER   Which cycle to run real for  [required]
  --cores INTEGER   Number of cores to use for real.exe
  --member INTEGER  When using different IC/BC for each member, which member
                    to process
  --help            Show this message and exit.

wrf-ensembly EXPERIMENT_PATH preprocess interpolate-chem¶

Uses interpolator-for-wrfchem to interpolate the chemical initial conditions onto the WRF domain.

Args: jobs: How many processes to use when interpolating the chemistry fields.

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess interpolate-chem [OPTIONS]

Options:

  --jobs INTEGER    Number of processes to use (also respects SLURM_NTASKS)
  --member INTEGER  When using different IC/BC for each member, which member
                    to process
  --help            Show this message and exit.

wrf-ensembly EXPERIMENT_PATH preprocess clean¶

Deletes the preprocessing directory and all its contents. Specifically removes: - One copy of WPS and WRF - Intermediate files (FILE_ and GRIBFILE.) - met_em files

Usage:

wrf-ensembly EXPERIMENT_PATH preprocess clean [OPTIONS]

Options:

  --help  Show this message and exit.

Observations¶

wrf-ensembly EXPERIMENT_PATH observations convert-obs¶

Converts observation files to DART obs_seq format (does not join per cycle)

Usage:

wrf-ensembly EXPERIMENT_PATH observations convert-obs [OPTIONS] [CYCLE]

Options:

  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH observations combine-obs¶

Joins converted observation files into a single obs_seq per cycle

Usage:

wrf-ensembly EXPERIMENT_PATH observations combine-obs [OPTIONS] [CYCLE]

Options:

  --jobs INTEGER RANGE  How many cycles to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH observations prepare-custom-window¶

Converts observation files to DART obs_seq format for a custom assimilation window

Usage:

wrf-ensembly EXPERIMENT_PATH observations prepare-custom-window 
    [OPTIONS] [[%Y-%m-%d|%Y-%m-%dT%H:%M:%S|%Y-%m-%d %H:%M:%S]] OUTPUT_PATH

Options:

  -l, --length INTEGER
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH observations obs-seq-to-nc¶

Convert the given obs_seq file to netCDF format

Usage:

wrf-ensembly EXPERIMENT_PATH observations obs-seq-to-nc [OPTIONS] OBS_SEQ_PATH
                                                        NC_PATH

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH observations preprocess-for-wrf¶

Runs all obs_seq files through the WRF preprocessing utility to remove observations outside the domain and optionally increase obs. errors near the boundary. If and how much is this error increase can be configured inside the observations config group.

Usage:

wrf-ensembly EXPERIMENT_PATH observations preprocess-for-wrf 
    [OPTIONS]

Options:

  --backup / --no-backup  Whether to backup the original `obs` directory to
                          `obs.tar.gz`. Existing backup is removed.
  --help                  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH observations list-files¶

Lists observation files that match the given cycle's assimilation window

Usage:

wrf-ensembly EXPERIMENT_PATH observations list-files [OPTIONS] CYCLE

Options:

  --write-to-file PATH  Write filenames to the specified text file (one per
                        line)
  --help                Show this message and exit.

Ensemble Management¶

wrf-ensembly EXPERIMENT_PATH ensemble setup¶

Copies initial/boundary conditions for each member.

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble setup [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble setup-from-other-experiment¶

Setup the ensemble using a forecast from another experiment.

The usecase for this command is having a control experiment that starts earlier, for spin-up reasons. You can initialise a second experiment from a mid-point and work forwards.

The other experiment must have the same cycle setup (start/end dates, output interval, boundary conditions interval) and the same domain. The forecast files for the requested cycle must be available in the scratch directory.

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble setup-from-other-experiment 
    [OPTIONS] OTHER_EXPERIMENT

Options:

  --cycle TEXT  Which cycle to use for initialisation  [required]
  --help        Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble generate-perturbations¶

Generates perturbations for all experiment cycles

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble generate-perturbations 
    [OPTIONS]

Options:

  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble apply-perturbations¶

Applies perturbations to the initial conditions of the current cycle. Make sure to update the boundary conditions afterwards!

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble apply-perturbations [OPTIONS]

Options:

  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble update-bc¶

Runs update_wrf_bc for all members to update boundary conditions. Use this after you have modified the initial conditions (perts or cycling).

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble update-bc [OPTIONS]

Options:

  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble advance-member¶

Advances the given MEMBER 1 cycle by running the model

You can control how many cores to use with --cores. If omitted, will check for SLURM_NTASKS in the environment and use that. If missing, will use 1 core.

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble advance-member [OPTIONS]

Options:

  --member INTEGER  Which member to advance  [required]
  --cores INTEGER   Number of cores to use for wrf.exe.
  --help            Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble filter¶

Runs the assimilation filter for the current cycle

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble filter [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble analysis¶

Combines the DART output files and the forecast to create the analysis.

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble analysis [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH ensemble cycle¶

Prepares the experiment for the next cycle by copying the cycled variables from the analysis to the initial conditions and preparing the namelist.

Usage:

wrf-ensembly EXPERIMENT_PATH ensemble cycle [OPTIONS]

Options:

  --use-forecast        Cycle with the latest forecast instead of the analysis
  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

Postprocessing¶

wrf-ensembly EXPERIMENT_PATH postprocess print-variables-to-keep¶

Prints which variables will be kept in the wrfout files after postprocessing. This is useful to check if the variables you want to keep are actually kept or check how the regex filters are applied.

The variables are defined in the PostprocessConfig:variables_to_keep variable of the configuration file.

Cycle 0 output for member 0 must exist for this command to work.

Usage:

wrf-ensembly EXPERIMENT_PATH postprocess print-variables-to-keep 
    [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH postprocess process-pipeline¶

Apply the configured data processor pipeline to output files.

This command replaces apply-scripts with a more efficient plugin-based approach that processes data in memory using a configurable pipeline of DataProcessor instances.

Usage:

wrf-ensembly EXPERIMENT_PATH postprocess process-pipeline [OPTIONS]

Options:

  --cycle INTEGER       Cycle to compute statistics for. Will compute for
                        current cycle if missing.
  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH postprocess statistics¶

Calculates the ensemble mean and standard deviation from the forecast/analysis files of given cycle. This function reads the *_post files created by the wrf_post and, optionally, apply_scripts commands.

Usage:

wrf-ensembly EXPERIMENT_PATH postprocess statistics [OPTIONS]

Options:

  --cycle INTEGER       Cycle to compute statistics for. Will compute for all
                        current cycle if missing.
  --jobs INTEGER RANGE  How many files to process in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH postprocess concatenate¶

Concatenates all output files (mean and standard deviation) into two files, one for analysis and one for forecast. It uses the _mean and _sd files created by the statistics command.

Usage:

wrf-ensembly EXPERIMENT_PATH postprocess concatenate [OPTIONS]

Options:

  --cycle INTEGER       Cycle to compute statistics for. Will compute for all
                        current cycle if missing.
  --jobs INTEGER RANGE  How many NCO commands to execute in parallel  [x>=0]
  --help                Show this message and exit.

wrf-ensembly EXPERIMENT_PATH postprocess clean¶

Clean up the scratch directory for the given cycle. Use after running the other postprocessing commands to save disk space.

Usage:

wrf-ensembly EXPERIMENT_PATH postprocess clean [OPTIONS]

Options:

  --cycle INTEGER  Cycle to clean up. Will clean for current cycle if missing.
  --remove-wrfout  Remove the raw wrfout files
  --help           Show this message and exit.

Status¶

wrf-ensembly EXPERIMENT_PATH status show¶

Prints the current experiment status with a rich table showing member status and experiment state

Usage:

wrf-ensembly EXPERIMENT_PATH status show [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH status runtime-stats¶

Prints the runtime statistics of all members across all cycles in a rich table

Usage:

wrf-ensembly EXPERIMENT_PATH status runtime-stats [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH status clear-runtime-stats¶

Clears all runtime statistics from the database

Usage:

wrf-ensembly EXPERIMENT_PATH status clear-runtime-stats [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH status reset¶

Resets the experiment state to the beginning (cycle 0, no advanced members)

Usage:

wrf-ensembly EXPERIMENT_PATH status reset [OPTIONS]

Options:

  --confirm  Confirm the reset action
  --help     Show this message and exit.

wrf-ensembly EXPERIMENT_PATH status set-member¶

Set the advanced status for a specific member by index

Usage:

wrf-ensembly EXPERIMENT_PATH status set-member [OPTIONS] MEMBER_INDEX ADVANCED

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH status set-all-members¶

Set the advanced status for all members

Usage:

wrf-ensembly EXPERIMENT_PATH status set-all-members [OPTIONS] ADVANCED

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH status set-experiment¶

Set the experiment state (cycle, filter run, analysis run)

Usage:

wrf-ensembly EXPERIMENT_PATH status set-experiment [OPTIONS]

Options:

  --cycle INTEGER         Set current cycle number
  --filter-run BOOLEAN    Set filter run status
  --analysis-run BOOLEAN  Set analysis run status
  --help                  Show this message and exit.

SLURM¶

wrf-ensembly EXPERIMENT_PATH slurm preprocessing¶

Creates a jobfile for running all preprocessing steps. Useful if you want to run WPS and real on your processing nodes.

Usage:

wrf-ensembly EXPERIMENT_PATH slurm preprocessing [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH slurm advance-members¶

Create a SLURM jobfile to advance each member of the ensemble

Usage:

wrf-ensembly EXPERIMENT_PATH slurm advance-members [OPTIONS]

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH slurm make-analysis¶

Creates a SLURM jobfile for the filter, analysis and cycle steps. At runtime, the job script will check whether there are observations available for the current cycle and will only run filter and analysis if they are found. Otherwise, only cycle will be run with the --use-forecast flag.

Usage:

wrf-ensembly EXPERIMENT_PATH slurm make-analysis [OPTIONS] CYCLE

Options:

  --help  Show this message and exit.

wrf-ensembly EXPERIMENT_PATH slurm postprocess¶

Create a SLURM jobfile to postprocess the WRF output

Usage:

wrf-ensembly EXPERIMENT_PATH slurm postprocess [OPTIONS] CYCLE

Options:

  --clean-scratch  Requires --run-postprocess. If set, the individual member's
                   forecasts are deleted from the scratch directories
  --help           Show this message and exit.

wrf-ensembly EXPERIMENT_PATH slurm queue-all-postprocessing¶

Queue postprocessing for all cycles of the experiment

Usage:

wrf-ensembly EXPERIMENT_PATH slurm queue-all-postprocessing 
    [OPTIONS]

Options:

  --clean-scratch        Requires --run-postprocess. If set, the individual
                         member's forecasts are deleted from the scratch
                         directories
  --first-cycle INTEGER  Queue postprocessing for all cycles starting from
                         this one
  --last-cycle INTEGER   Queue postprocessing for all cycles up to this one
  --help                 Show this message and exit.

wrf-ensembly EXPERIMENT_PATH slurm run-experiment¶

Creates jobfiles for all experiment steps and queues them in the correct order. This does not deal with the initial steps (setup, initial/boundary conditions, ...), only the member advancing, analysis and cycling. Postprocessing will be queued if you use --run_postprocess.

If for some cycle there are not prepared observations (in the obs directory), the generated job will skip the analysis step and go straight to cycling.

Usage:

wrf-ensembly EXPERIMENT_PATH slurm run-experiment [OPTIONS]

Options:

  --all-cycles / --next-cycle-only
                                  After the current cycle, automatically queue
                                  next cycle, until experiment is over
  --run-postprocess               Compute statistics for the current cycle
                                  after the analysis step
  --clean-scratch                 Requires --run-postprocess. If set, the
                                  individual member's forecasts are deleted
                                  from the scratch directories
  --only-advance                  Only queue the advance steps
  --run-until INTEGER             Run until this cycle (end-inclusive)
  --help                          Show this message and exit.