Skip to content

Allow -with-docker to be used without a default container image#7054

Merged
bentsherman merged 1 commit intomasterfrom
fix-with-docker
Apr 22, 2026
Merged

Allow -with-docker to be used without a default container image#7054
bentsherman merged 1 commit intomasterfrom
fix-with-docker

Conversation

@bentsherman
Copy link
Copy Markdown
Member

This PR allows the -with-docker option to be used without a default container image.

Particularly useful when running modules directly:

nextflow module run nf-core/fastqc --meta.id 1 --reads sample.fastq -with-docker

The above currently fails because Nextflow currently requires a default container image, even if every process defines its own container.

Whereas -with-conda works out of the box, because it doesn't require a default conda environment:

nextflow module run nf-core/fastqc --meta.id 1 --reads sample.fastq -with-conda

The consequence of this PR is that when a process doesn't define a container image, it will implicitly run without containerization. This is consistent with Conda: processes that don't define a conda environment will simply run without conda.

@netlify
Copy link
Copy Markdown

netlify Bot commented Apr 20, 2026

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit cc41afa
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69e8dab08f17bf0008015bc8

Copy link
Copy Markdown
Contributor

@jorgee jorgee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doing some tests I have seen that when using -with-docker but the container image is not provided, it runs without Docker without any warning, and it fails because a command is not found. Moreover, if a retry is set, it will be repeated several times.

I was wondering if a check would be needed at submission time to avoid running tasks when containers are enabled and no image is provided. I have created PR #7061 to include this check. Let me know if you agree or if I missed something.

@bentsherman
Copy link
Copy Markdown
Member Author

Like I said in the PR description -- I think this is consistent with Conda. Processes that don't define a Conda environment will simply run without Conda

I think this is also what happens if you set docker.enabled = true in the config instead of -with-docker on the command line. The global default check was only implemented by -with-docker

@bentsherman
Copy link
Copy Markdown
Member Author

Re-implementing the check at the process level as you did with #7061 , I think would actually be a breaking change, because it is currently possible to mix containerized with non-containerized tasks in a run using docker.enabled = true instead of -with-docker.

@jorgee
Copy link
Copy Markdown
Contributor

jorgee commented Apr 21, 2026

Ok, understood. I'll close #7061

Signed-off-by: Ben Sherman <bentshermann@gmail.com>
@pditommaso
Copy link
Copy Markdown
Member

What happens if not container is defined?

@bentsherman
Copy link
Copy Markdown
Member Author

@pditommaso it will behave the same as setting docker.enabled = true in config. Processes with no container will simply run without containerization

@bentsherman bentsherman merged commit 41759d3 into master Apr 22, 2026
24 checks passed
@bentsherman bentsherman deleted the fix-with-docker branch April 22, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants