Skip to content

Commit dcb34a0

Browse files
committed
Adding apple-container as a new executor
1 parent da06e9a commit dcb34a0

14 files changed

Lines changed: 699 additions & 2 deletions

File tree

docs/container.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,71 @@
44

55
Nextflow supports a variety of container runtimes. Containerization allows you to write self-contained and truly reproducible computational pipelines, by packaging the binary dependencies of a script into a standard and portable format that can be executed on any platform that supports a container runtime. Furthermore, the same pipeline can be transparently executed with any of the supported container runtimes, depending on which runtimes are available in the target compute environment.
66

7+
(container-apple)=
8+
9+
## Apple containers
10+
11+
:::{versionadded} 26.04.0-edge
12+
:::
13+
14+
[Apple containers](https://github.com/apple/containerization) is a container runtime for Apple Silicon Macs that runs OCI-compatible container images natively using the macOS Virtualization framework. It provides fast, lightweight container execution without requiring Docker Desktop or any third-party virtualization software.
15+
16+
### Prerequisites
17+
18+
Apple containers requires macOS 26 or later and Apple Silicon (M-series chip). Install the `container` CLI tool from Apple and ensure it is available on your `PATH`.
19+
20+
### How it works
21+
22+
To run your pipeline with Apple containers, use the `-with-apple-container` command line option:
23+
24+
```bash
25+
nextflow run <your script> -with-apple-container [OCI container image]
26+
```
27+
28+
Alternatively, enable the engine in your configuration file:
29+
30+
```groovy
31+
process.container = 'ubuntu:22.04'
32+
apple.enabled = true
33+
```
34+
35+
Every time a process is executed, Nextflow will wrap it in a `container run` command using the image specified by the `container` directive.
36+
37+
### Architecture and Rosetta
38+
39+
Apple containers runs `linux/arm64` images by default. To run `linux/amd64` images transparently via [Rosetta](https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment), set the `arch` directive on the relevant processes:
40+
41+
```groovy
42+
process {
43+
arch = 'x86_64'
44+
}
45+
```
46+
47+
Nextflow will automatically add `--platform linux/amd64 --rosetta` to the `container run` command. For pipelines that ship native arm64 images, no `arch` directive is needed.
48+
49+
### Configuration
50+
51+
```groovy
52+
process.container = 'ubuntu:22.04'
53+
54+
apple {
55+
enabled = true
56+
}
57+
```
58+
59+
Or use the dedicated `apple_container` executor to make the engine selection explicit:
60+
61+
```groovy
62+
process {
63+
executor = 'apple_container'
64+
container = 'ubuntu:22.04'
65+
}
66+
```
67+
68+
### Advanced settings
69+
70+
Apple container advanced configuration settings are described in {ref}`config-apple` section in the Nextflow configuration page.
71+
772
:::{note}
873
When creating a container image to use with Nextflow, make sure that Bash (3.x or later) and `ps` are installed in the image, along with other tools required for collecting metrics (See {ref}`this section <execution-report-tasks>`). Bash should be available on the path `/bin/bash` and it should be the container entrypoint.
974
:::

docs/reference/config.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -480,6 +480,38 @@ The following settings are available:
480480
`azure.storage.tokenDuration`
481481
: The duration of the SAS token generated by Nextflow when the `sasToken` option is *not* specified (default: `48h`).
482482

483+
(config-apple)=
484+
485+
## `apple`
486+
487+
The `apple` scope controls how [Apple containers](https://github.com/apple/containerization) are executed by Nextflow.
488+
489+
The following settings are available:
490+
491+
`apple.enabled`
492+
: Execute tasks with Apple containers (default: `false`).
493+
494+
`apple.engineOptions`
495+
: Specify additional options supported by the Apple container engine i.e. `container [OPTIONS]`.
496+
497+
`apple.envWhitelist`
498+
: Comma separated list of environment variable names to be included in the container environment.
499+
500+
`apple.kill`
501+
: How the container should be stopped. Use `true` to send a SIGTERM (default), `false` to skip, or a signal name e.g. `'SIGKILL'` to use a specific signal.
502+
503+
`apple.registry`
504+
: The registry from where container images are pulled. It should be only used to specify a private registry server. It should NOT include the protocol prefix i.e. `http://`.
505+
506+
`apple.remove`
507+
: Clean-up the container after the execution (default: `true`).
508+
509+
`apple.runOptions`
510+
: Specify extra command line options supported by the `container run` command.
511+
512+
`apple.temp`
513+
: Mounts a path of your choice as the `/tmp` directory in the container. Use the special value `'auto'` to create a temporary directory each time a container is created.
514+
483515
(config-charliecloud)=
484516

485517
## `charliecloud`

modules/nextflow/src/main/groovy/nextflow/Session.groovy

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ import nextflow.cache.CacheDB
3737
import nextflow.cache.CacheFactory
3838
import nextflow.conda.CondaConfig
3939
import nextflow.config.Manifest
40+
import nextflow.container.AppleContainerConfig
4041
import nextflow.container.ApptainerConfig
4142
import nextflow.container.CharliecloudConfig
4243
import nextflow.container.ContainerConfig
@@ -1187,6 +1188,7 @@ class Session implements ISession {
11871188
new SingularityConfig(config.singularity as Map ?: Collections.emptyMap()),
11881189
new ApptainerConfig(config.apptainer as Map ?: Collections.emptyMap()),
11891190
new CharliecloudConfig(config.charliecloud as Map ?: Collections.emptyMap()),
1191+
new AppleContainerConfig(config.apple as Map ?: Collections.emptyMap()),
11901192
] as List<ContainerConfig>
11911193

11921194
if( engine ) {

modules/nextflow/src/main/groovy/nextflow/cli/CmdRun.groovy

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -198,6 +198,9 @@ class CmdRun extends CmdBase implements HubOptions {
198198
@Parameter(names = '-with-charliecloud', description = 'Enable process execution in a Charliecloud container runtime')
199199
def withCharliecloud
200200

201+
@Parameter(names = '-with-apple-container', description = 'Enable process execution in an Apple container runtime')
202+
def withApple
203+
201204
@Parameter(names = '-with-singularity', description = 'Enable process execution in a Singularity container')
202205
def withSingularity
203206

modules/nextflow/src/main/groovy/nextflow/cli/Launcher.groovy

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -294,6 +294,10 @@ class Launcher {
294294
normalized << '-'
295295
}
296296

297+
else if( current == '-with-apple-container' && (i==args.size() || args[i].startsWith('-'))) {
298+
normalized << '-'
299+
}
300+
297301
else if( current == '-with-conda' && (i==args.size() || args[i].startsWith('-'))) {
298302
normalized << '-'
299303
}

modules/nextflow/src/main/groovy/nextflow/config/ConfigBuilder.groovy

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -778,6 +778,10 @@ class ConfigBuilder {
778778
if( cmdRun.withCharliecloud ) {
779779
configContainer(config, 'charliecloud', cmdRun.withCharliecloud)
780780
}
781+
782+
if( cmdRun.withApple ) {
783+
configContainer(config, 'apple', cmdRun.withApple)
784+
}
781785
}
782786

783787
private void configContainer(ConfigObject config, String engine, def cli) {
Lines changed: 178 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,178 @@
1+
/*
2+
* Copyright 2013-2026, Seqera Labs
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
package nextflow.container
17+
18+
import groovy.transform.CompileStatic
19+
20+
/**
21+
* Helper methods to handle Apple containers.
22+
*
23+
* Wraps tasks in {@code container run} commands using the Apple container CLI,
24+
* which runs linux/arm64 images natively on Apple Silicon via the Virtualization
25+
* framework. For amd64 images, {@code --rosetta} is added automatically.
26+
*
27+
* @author Joon-Klaps
28+
*/
29+
@CompileStatic
30+
class AppleContainerBuilder extends ContainerBuilder<AppleContainerBuilder> {
31+
32+
private boolean remove
33+
34+
private String registry
35+
36+
private String name
37+
38+
private String removeCommand
39+
40+
private String killCommand
41+
42+
private Object kill = true
43+
44+
AppleContainerBuilder(String name, AppleContainerConfig config) {
45+
this.image = name
46+
47+
if( config.engineOptions )
48+
addEngineOptions(config.engineOptions)
49+
50+
this.remove = config.remove
51+
52+
if( config.registry )
53+
this.registry = config.registry
54+
55+
if( config.runOptions )
56+
addRunOptions(config.runOptions)
57+
58+
if( config.temp )
59+
this.temp = config.temp
60+
}
61+
62+
AppleContainerBuilder(String name) {
63+
this(name, new AppleContainerConfig([:]))
64+
}
65+
66+
@Override
67+
AppleContainerBuilder params( Map params ) {
68+
if( !params ) return this
69+
70+
if( params.containsKey('entry') )
71+
this.entryPoint = params.entry
72+
73+
if( params.containsKey('kill') )
74+
this.kill = params.kill
75+
76+
if( params.containsKey('readOnlyInputs') )
77+
this.readOnlyInputs = params.readOnlyInputs?.toString() == 'true'
78+
79+
return this
80+
}
81+
82+
@Override
83+
AppleContainerBuilder setName( String name ) {
84+
this.name = name
85+
return this
86+
}
87+
88+
@Override
89+
AppleContainerBuilder build(StringBuilder result) {
90+
assert image
91+
92+
result << 'container '
93+
94+
if( engineOptions )
95+
result << engineOptions.join(' ') << ' '
96+
97+
result << 'run -i '
98+
99+
// add the environment
100+
appendEnv(result)
101+
102+
if( temp )
103+
result << "-v $temp:/tmp "
104+
105+
// mount the input folders
106+
result << makeVolumes(mounts)
107+
result << '-w "$NXF_TASK_WORKDIR" '
108+
109+
if( entryPoint )
110+
result << '--entrypoint ' << entryPoint << ' '
111+
112+
if( runOptions )
113+
result << runOptions.join(' ') << ' '
114+
115+
if( cpus )
116+
result << "--cpus ${cpus} "
117+
118+
if( memory )
119+
result << "--memory ${memory.toUpperCase()} "
120+
121+
// --platform selects the image variant; --rosetta enables x86_64 execution via Rosetta.
122+
// Both are needed for amd64 images: without --platform the CLI defaults to linux/arm64.
123+
if( platform ) {
124+
result << "--platform ${platform} "
125+
if( !platform.contains("arm64") )
126+
result << '--rosetta '
127+
}
128+
129+
// the name is after the user option so it has precedence over any options provided by the user
130+
if( name )
131+
result << '--name ' << name << ' '
132+
133+
if( registry )
134+
result << registry
135+
136+
// finally the container image
137+
result << image
138+
139+
// return the run command as result
140+
runCommand = result.toString()
141+
142+
// use an explicit 'container rm' command after the container stops
143+
if( remove && name ) {
144+
removeCommand = 'container rm ' + name
145+
}
146+
147+
if( kill && name ) {
148+
killCommand = 'container stop '
149+
// if `kill` is a string it is interpreted as the kill signal
150+
if( kill instanceof String ) killCommand = "container kill --signal $kill "
151+
killCommand += name
152+
}
153+
154+
return this
155+
}
156+
157+
@Override
158+
String getRunCommand(String launcher) {
159+
if( !launcher )
160+
return getRunCommand()
161+
def result = getRunCommand()
162+
result += entryPoint ? " -c \"$launcher\"" : " $launcher"
163+
return result
164+
}
165+
166+
/**
167+
* @return The command string to remove a container
168+
*/
169+
@Override
170+
String getRemoveCommand() { removeCommand }
171+
172+
/**
173+
* @return The command string to stop/kill a running container
174+
*/
175+
@Override
176+
String getKillCommand() { killCommand }
177+
178+
}

0 commit comments

Comments
 (0)