Directives
Directives are optional settings that affect the execution of the current process. They are declared at the top of a process definition, before any input, output, or script section.
By default, directives are evaluated when the process is defined. However, if the value is a dynamic string or closure, it will be evaluated separately for each task. See Dynamic directives for more information.
Directives can also be specified in a configuration file using the process config scope, which is the recommended way to configure a pipeline for a particular compute environment. See Process configuration for more information.
Some directives are only supported by specific executors. See Executors for more information about each executor.
The following directives are available:
| Directive | Description |
|---|---|
accelerator | The number of hardware accelerators (e.g. GPUs) required by each task. |
afterScript | A Bash snippet to execute after the task script. |
arch | The CPU architecture to build the software required by each task. |
array | Submit tasks as job arrays, for executors that support it. |
beforeScript | A Bash snippet to execute before the task script. |
cache | Whether and how task executions are cached. |
clusterOptions | Additional submission options for grid executors. |
conda | The set of Conda packages required by each task. |
container | The container image required by each task. |
containerOptions | Additional options for the underlying container runtime. |
cpus | The number of CPUs required by each task. |
debug | Print the standard output of each task to the pipeline standard output. |
disk | The amount of disk storage required by each task. |
errorStrategy | How to handle task failures. |
executor | The underlying system where tasks are executed. |
ext | A generic directive for user-defined properties. |
fair | Emit process outputs in the order in which they were received. |
hints | Executor-specific hints, specified as key-value pairs. |
label | A custom label, used to apply shared process configuration. |
machineType | A predefined machine type for cloud executors that support it. |
maxErrors | The maximum number of task failures allowed for a process. |
maxForks | The maximum number of concurrent task executions for a process. |
maxRetries | The maximum number of times a task can be retried. |
maxSubmitAwait | How long a task can remain in the submission queue before failing. |
memory | The amount of memory required by each task. |
module | The set of Environment Modules required by each task. |
penv | The parallel environment to use when submitting tasks to SGE. |
pod | Pod-specific settings for the Kubernetes executor. |
publishDir | Publish matching process output files to a target directory. |
queue | The queue to which tasks should be submitted. |
resourceLabels | Custom name-value pairs to attach to task executions. |
resourceLimits | Environment-specific limits for task resource requests. |
scratch | Execute each task in a temporary directory local to the compute node. |
secret | The secrets required by each task. |
shell | A custom shell command for process scripts. |
spack | The set of Spack packages required by each task. |
stageInMode | How input files are staged into the task directory. |
stageOutMode | How output files are staged out of the scratch directory. |
storeDir | Store task outputs in a permanent store directory. |
tag | A custom identifier for each task execution. |
time | The maximum runtime for each task. |
See also
- Dynamic directives: How to define a directive value based on the task inputs.
- Dynamic task resources: How to define task resources that scale with each retry.
- Task directive values: How to access directive values in a process body.