You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/config.md
+51-80Lines changed: 51 additions & 80 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,50 +2,48 @@
2
2
3
3
# Configuration
4
4
5
-
## Configuration file
5
+
## Configuration files
6
6
7
-
When a pipeline script is launched, Nextflow looks for configuration files in multiple locations. Since each configuration file may contain conflicting settings, they are applied in the following order (from lowest to highest priority):
7
+
When you launch a pipeline script, Nextflow detects configuration files from multiple sources and applies them in the following order (from lowest to highest priority):
8
8
9
-
1.The config file `$HOME/.nextflow/config` (or `$NXF_HOME/config` when {ref}`NXF_HOME <nxf-env-vars>` is set).
10
-
2.The config file `nextflow.config` in the project directory
11
-
3.The config file `nextflow.config` in the launch directory
12
-
4. Config files specified using the `-c <config-files>` option
9
+
1.`$NXF_HOME/config` (defaults to `$HOME/.nextflow/config`)
10
+
2.`nextflow.config` in the project directory
11
+
3.`nextflow.config` in the launch directory
12
+
4. Config files specified with `-c <config-files>`
13
13
14
14
:::{tip}
15
-
You can alternatively use the `-C <config-file>` option to specify a fixed set of configuration files and ignore all other files.
15
+
You can use the `-C <config-file>` option to specify a fixed set of configuration files and ignore all other files.
16
16
:::
17
17
18
18
(config-syntax)=
19
19
20
20
## Syntax
21
21
22
-
The Nextflow configuration syntax is based on the Nextflow script syntax. It is designed for setting configuration options in a declarative manner while also allowing for dynamic expressions where appropriate.
22
+
Nextflow configuration uses a similar syntax as Nextflow scripts. Configuration options must be set declaratively, but the value of a config option can be an arbitrary expression.
23
23
24
-
A Nextflow config file may consist of any number of *assignments*, *blocks*, and *includes*. Config files may also contain comments in the same manner as scripts.
25
-
26
-
See {ref}`syntax-page` for more information about the Nextflow script syntax.
24
+
A config file may contain any number of [assignments](#assignments), [blocks](#blocks), and [includes](#includes). You can add comments just like in scripts.
27
25
28
26
### Assignments
29
27
30
-
A config assignment consists of a config option and an expression separated by an equals sign:
28
+
A config assignment sets a config option to a value:
31
29
32
30
```groovy
33
31
workDir = 'work'
34
32
docker.enabled = true
35
33
process.maxErrors = 10
36
34
```
37
35
38
-
A config option consists of an *option name* prefixed by any number of *scopes* separated by dots. Config scopes are used to group related config options. See {ref}`config-options` for the full set of config options.
36
+
The config option consists of an *option name* prefixed by any number of *scopes*. Scopes group related config options. See {ref}`config-options` for the full set of config options.
39
37
40
-
The expression is typically a literal value such as a number, boolean, or string. However, any expression can be used:
38
+
The value is typically a literal value, such as a number, boolean, or string. However, you can use any {ref}`expression <syntax-expressions>`:
Relative paths are resolved against the location of the including file.
91
89
92
90
:::{note}
93
-
Config includes can also be specified within config blocks. However, config files should only be included at the top level or in a [profile](#config-profiles) so that the included config file is valid on its own and in the context in which it is included.
91
+
Config files should only be included at the top level or in a [profile](#config-profiles). This ensures the included config file is valid on its own and in the context in which it is included.
94
92
:::
95
93
96
-
(config-constants)=
97
-
98
-
## Constants
94
+
## Constants and functions
99
95
100
96
The following constants are globally available in a Nextflow configuration file:
101
97
@@ -113,20 +109,18 @@ The following constants are globally available in a Nextflow configuration file:
113
109
`secrets: Map<String,String>`
114
110
: Map of pipeline secrets. See {ref}`secrets-page` for more information.
115
111
116
-
## Functions
117
-
118
112
The following functions are globally available in a Nextflow configuration file:
119
113
120
114
`env( name: String ) -> String`
121
-
: :::{versionadded} 24.11.0-edge
115
+
: :::{versionadded} 25.04.0
122
116
:::
123
117
: Get the value of the environment variable with the specified name in the Nextflow launch environment.
124
118
125
119
(config-params)=
126
120
127
121
## Parameters
128
122
129
-
Pipeline parameters can be defined in the config file using the `params` scope:
123
+
You can define pipeline parameters in config files using the `params` scope:
130
124
131
125
```groovy
132
126
// dot syntax
@@ -141,10 +135,10 @@ params {
141
135
```
142
136
143
137
:::{note}
144
-
When including a config file, the included config is evaluated with the parameters that are defined before the include. Parameters defined after the include are not visible to the included config.
138
+
When including a config file, the included config is evaluated with the parameters defined before the include. The included config cannot see parameters defined after the include.
145
139
:::
146
140
147
-
As a best practice, declare parameters in the config file only if they are used by other config options. If a parameter is used within the script, declare it there and override it in config profiles as needed. For example:
141
+
You should declare parameters in the config file only when other config options use them. When you use a parameter in the script, you should declare it there and override it in config profiles as needed:
148
142
149
143
```nextflow
150
144
// main.nf
@@ -166,13 +160,13 @@ profiles {
166
160
}
167
161
```
168
162
169
-
See {ref}`cli-params` for information about how pipeline parameters are resolved at runtime.
163
+
See {ref}`cli-params` for information about how Nextflow resolves pipeline parameters at runtime.
170
164
171
165
(config-process)=
172
166
173
167
## Process configuration
174
168
175
-
The `process` scope allows you to specify {ref}`process directives <process-reference>` separately from the pipeline code.
169
+
You can use the `process` scope to specify {ref}`process directives <process-reference>` separately from the pipeline code.
176
170
177
171
For example:
178
172
@@ -184,13 +178,13 @@ process {
184
178
}
185
179
```
186
180
187
-
By using this configuration, all processes in your pipeline will be executed through the SGE cluster, with the specified settings.
181
+
This configuration executes all processes using the SGE executor with the given settings.
188
182
189
183
(config-process-selectors)=
190
184
191
185
### Process selectors
192
186
193
-
The `withLabel`selectors allow the configuration of all processes annotated with a {ref}`process-label` directive as shown below:
187
+
You can use the `withLabel`selector to configure all processes annotated with a {ref}`process-label` directive:
194
188
195
189
```groovy
196
190
process {
@@ -202,9 +196,9 @@ process {
202
196
}
203
197
```
204
198
205
-
The above configuration example assigns 16 cpus, 64 Gb of memory and the `long` queue to all processes annotated with the `big_mem` label.
199
+
This configuration assigns 16 CPUs, 64 GB of memory, and the `long` queue to all processes with the `big_mem` label.
206
200
207
-
In the same manner, the `withName` selector allows the configuration of a specific process in your pipeline by its name. For example:
201
+
You can use the `withName` selector to configure a specific process by its name:
208
202
209
203
```groovy
210
204
process {
@@ -216,19 +210,22 @@ process {
216
210
}
217
211
```
218
212
219
-
The `withName` selector applies both to processes defined with the same name and processes included under the same alias. For example, `withName: hello` will apply to any process originally defined as `hello`, as well as any process included under the alias `hello`.
213
+
The `withName` selector matches both:
220
214
221
-
Furthermore, selectors for the alias of an included process take priority over selectors for the original name of the process. For example, given a process defined as `hello` and included as `sayHello`, the selectors `withName: hello` and `withName: sayHello` will both be applied to the process, with the second selector taking priority over the first.
215
+
- Processes defined with that name
216
+
- Processes included under that alias
217
+
218
+
When you include a process with an alias, selectors for the alias take priority over selectors for the original name. For example, if you define a process as `hello` and include it as `sayHello`, both `withName: hello` and `withName: sayHello` apply, with `withName: sayHello` taking priority.
222
219
223
220
:::{tip}
224
-
Label and process names do not need to be enclosed with quotes, provided the name does not include special characters (`-`, `!`, etc) and is not a keyword or a built-in type identifier. When in doubt, you can enclose the label name or process name with single or double quotes.
221
+
You don't need to enclose label and process names in quotes unless they contain special characters (`-`, `!`, etc.) or are keywords or built-in type identifiers.
225
222
:::
226
223
227
224
(config-selector-expressions)=
228
225
229
226
### Selector expressions
230
227
231
-
Both label and process name selectors allow the use of a regular expression in order to apply the same configuration to all processes matching the specified pattern condition. For example:
228
+
You can use regular expressions in process selectors to apply the same configuration to all processes matching the pattern:
232
229
233
230
```groovy
234
231
process {
@@ -239,9 +236,9 @@ process {
239
236
}
240
237
```
241
238
242
-
The above configuration snippet requests 2 cpus and 4 GB of memory for processes labeled as `hello` or `bye`.
239
+
This configuration requests 2 CPUs and 4 GB of memory for processes labeled as `hello` or `bye`.
243
240
244
-
A process selector can be negated prefixing it with the special character `!`. For example:
241
+
You can negate a selector expression by prefixing it with the special character `!`:
245
242
246
243
```groovy
247
244
process {
@@ -251,13 +248,13 @@ process {
251
248
}
252
249
```
253
250
254
-
The above configuration snippet sets 2 cpus for every process labeled as `hello` and 4 cpus to every process*not* labeled as `hello`. It also specifies the `long` queue for every process whose name does *not* start with `align`.
251
+
This configuration sets 2 CPUs for processes labeled as `hello` and 4 CPUs for all processes*not* labeled as `hello`. It also specifies the `long` queue for processes whose name does *not* start with `align`.
255
252
256
253
(config-selector-priority)=
257
254
258
255
### Selector priority
259
256
260
-
Process configuration settings are applied to a process in the following order (from lowest to highest priority):
257
+
Nextflow applies process configuration settings in the following order (from lowest to highest priority):
261
258
262
259
1. Process configuration settings (without a selector)
263
260
2. Process directives in the process definition
@@ -277,17 +274,18 @@ process {
277
274
}
278
275
```
279
276
280
-
With the above configuration:
281
-
- All processes will use 4 cpus (unless otherwise specified in their process definition).
282
-
- Processes annotated with the `hello` label will use 8 cpus.
283
-
- Any process named `bye` (or imported as `bye`) will use 16 cpus.
284
-
- Any process named `bye` (or imported as `bye`) invoked by a workflow named `aloha` will use 32 cpus.
277
+
This configuration:
278
+
279
+
- Sets 4 CPUs for all processes (unless otherwise specified in their process definition)
280
+
- Sets 8 CPUs for processes annotated with the `hello` label
281
+
- Sets 16 CPUs for any process named `bye` (or imported as `bye`)
282
+
- Sets 32 CPUs for any process named `bye` (or imported as `bye`) invoked by a workflow named `aloha`
285
283
286
284
(config-profiles)=
287
285
288
286
## Config profiles
289
287
290
-
Configuration files can define one or more *profiles*. A profile is a set of configuration settings that can be selected during pipeline execution using the `-profile` command line option.
288
+
Configuration files can define one or more *profiles*. A profile is a set of configuration settings that can be selected at runtime using the `-profile` command line option.
291
289
292
290
Configuration profiles are defined in the `profiles` scope. For example:
293
291
@@ -311,44 +309,18 @@ profiles {
311
309
}
312
310
```
313
311
314
-
The above configuration defines three profiles: `standard`, `cluster`, and `cloud`. Each profile provides a different configuration for a given execution environment. The `standard` profile is used by default when no profile is specified.
312
+
This configuration defines three profiles: `standard`, `cluster`, and `cloud`. Each profile provides a different configuration for a given execution environment. When you do not specify a profile, Nextflow uses the `standard`profile by default.
315
313
316
-
Configuration profiles can be specified at runtime as a comma-separated list:
314
+
You can enable configuration profiles at runtime as a comma-separated list:
317
315
318
316
```bash
319
-
nextflow run <your script> -profile standard,cloud
320
-
```
321
-
322
-
Config profiles are applied in the order in which they were defined in the config file, regardless of the order they are specified on the command line.
323
-
324
-
:::{versionadded} 25.02.0-edge
325
-
When using the {ref}`strict config syntax <updating-config-syntax>`, profiles are applied in the order in which they are specified on the command line.
326
-
:::
327
-
328
-
:::{danger}
329
-
When defining a profile in the config file, avoid using both the dot and block syntax for the same scope. For example:
330
-
331
-
```groovy
332
-
profiles {
333
-
cluster {
334
-
process.memory = '2 GB'
335
-
process {
336
-
cpus = 2
337
-
}
338
-
}
339
-
}
317
+
nextflow run main.nf -profile standard,cloud
340
318
```
341
319
342
-
Due to a limitation of the legacy config parser, the first setting will be overwritten by the second:
343
-
344
-
```console
345
-
$ nextflow config -profile cluster
346
-
process {
347
-
cpus = 2
348
-
}
349
-
```
320
+
Nextflow applies config profiles in the order in which they were defined in the config, regardless of the order you specify them on the command line.
350
321
351
-
This limitation can be avoided by using the {ref}`strict config syntax <updating-config-syntax>`.
322
+
:::{versionadded} 25.04.0
323
+
When using the {ref}`strict syntax <strict-syntax-page>`, Nextflow applies profiles in the order you specify them on the command line.
352
324
:::
353
325
354
326
(config-workflow-handlers)=
@@ -363,7 +335,6 @@ You can define workflow event handlers in the config file:
Copy file name to clipboardExpand all lines: docs/reference/syntax.md
+2-1Lines changed: 2 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -602,6 +602,8 @@ catch( IOException e ) {
602
602
603
603
The try block will be executed, and if an error is raised and matches the expected error type of a catch clause, the code in that catch clause will be executed. If no catch clause is matched, the error will be raised to the next enclosing try/catch statement, or to the Nextflow runtime.
604
604
605
+
(syntax-expressions)=
606
+
605
607
## Expressions
606
608
607
609
An expression represents a value. A *literal* value is an expression whose value is known at compile-time, such as a number, string, or boolean. All other expressions must be evaluated at run-time.
@@ -1010,4 +1012,3 @@ The following legacy features were excluded from this page because they are depr
1010
1012
- The `addParams` and `params` clauses of include declarations. See {ref}`module-params` for more information.
1011
1013
- The `when:` section of a process definition. See {ref}`process-when` for more information.
1012
1014
- The `shell:` section of a process definition. See {ref}`process-shell` for more information.
1013
-
- The implicit `it` closure parameter. See {ref}`script-closure` for more information.
0 commit comments