Skip to main content

publishDir

note

Workflow outputs can be used instead of publishDir. See Migrating to workflow outputs to learn how to migrate existing code.

The publishDir directive publishes matching process output files to a target directory. For example:

process hello {
    publishDir '/data/chunks'

    output:
    path 'chunk_*'

    script:
    """
    printf 'Hola' | split -b 1 - chunk_
    """
}

The above example publishes the chunk_* output files into the /data/chunks directory.

Only files that match the declaration in the output block are published, not all the outputs of the process.

The publishDir directive can be specified more than once in order to publish output files to different target directories based on different rules.

By default, files are published via symbolic link from the task directory to the target directory. Use the mode option to control this behavior:

process hello {
    publishDir '/data/chunks', mode: 'copy', overwrite: false

    output:
    path 'chunk_*'

    script:
    """
    printf 'Hola' | split -b 1 - chunk_
    """
}
warning

Output files are published asynchronously after the task execution, so they may not be immediately available in the publish directory during the pipeline run. Downstream processes should access output files through the declared process outputs, not the publish directory.

Available options:

contentType​

Experimental: currently only supported for S3.

Allow specifying the media content type of the published file a.k.a. MIME type. If set to true, the content type is inferred from the file extension (default: false).

enabled​

Enable or disable the publish rule depending on the boolean value specified (default: true).

failOnError​
Changed in version 24.04

The default value was changed from false to true

When true abort the execution if some file can't be published to the specified target directory or bucket for any cause (default: true)

mode​

The file publishing method. Can be one of the following values:

  • 'copy': Copies the output files into the publish directory.
  • 'copyNoFollow': Copies the output files into the publish directory without following symlinks ie. copies the links themselves.
  • 'link': Creates a hard link in the publish directory for each output file.
  • 'move': Moves the output files into the publish directory. Note: this is only supposed to be used for a terminal process i.e. a process whose output is not consumed by any other downstream process.
  • 'rellink': Creates a relative symbolic link in the publish directory for each output file.
  • 'symlink': Creates an absolute symbolic link in the publish directory for each output file (default).
overwrite​

When true any existing file in the target directory will be overridden (default: true during normal pipeline execution and false when pipeline execution is resumed).

path​

Specifies the directory where files need to be published. Note: the syntax publishDir '/some/dir' is a shortcut for publishDir path: '/some/dir'.

pattern​

Specifies a glob file pattern that selects which files to publish from the overall set of output files.

saveAs​

A closure which, given the name of the file being published, returns the actual file name or a full path where the file is required to be stored. This can be used to rename or change the destination directory of the published files dynamically by using a custom strategy. Return the value null from the closure to not publish a file. This is useful when the process has multiple output files, but you want to publish only some of them.

storageClass​
Added in version 23.04

Experimental: currently only supported for S3.

Allow specifying the storage class to be used for the published file.

tags​

Experimental: currently only supported for S3.

Allow the association of arbitrary tags with the published file e.g. tags: [MESSAGE: 'Hello world'].

On this Page