Record types (#6679)

Author: bentsherman

adr/20260306-record-types.md

@@ -52,7 +52,7 @@ process fastqc {
 
     script:
     """
-    echo 'meta: ${meta}`
+    echo 'meta: ${meta}'
     echo 'fastq: ${fastq}'
     echo 'extra_args: ${extra_args}'
     """
@@ -65,6 +65,8 @@ All {ref}`standard types <stdlib-types>` except for the dataflow types (`Channel
 
 Nextflow automatically stages `Path` inputs and `Path` collections (such as `Set<Path>`) into the task directory.
 
+### Nullable inputs
+
 By default, tasks fail if any input receives a `null` value. To allow `null` values, add `?` to the type annotation:
 
 ```nextflow
@@ -85,10 +87,83 @@ process cat_opt {
 }
 ```
 
-### Stage directives
+### Record inputs
+
+Inputs with type `Record` can declare the name and type of each record field:
+
+```nextflow
+process fastqc {
+    input:
+    sample: Record {
+        id: String
+        fastq: Path
+    }
+
+    script:
+    """
+    echo 'id: ${sample.id}'
+    echo 'fastq: ${sample.fastq}'
+    """
+}
+```
+
+In this example, the record is staged into the task as `sample`, and `sample.fastq` is staged as an input file since the `fastq` field is declared with type `Path`.
+
+When the process is invoked, the incoming record should contain the specified fields, or else the run will fail. If the record has additional fields not declared by the process input, they are ignored.
+
+:::{tip}
+Record inputs are a useful way to select a subset of fields from a larger record. This way, the process only stages what it needs, allowing you to keep related data together in your workflow logic.
+:::
+
+You can achieve the same behavior using an external record type:
+
+```nextflow
+process fastqc {
+    input:
+    sample: Sample
+
+    script:
+    """
+    echo 'id: ${sample.id}'
+    echo 'fastq: ${sample.fastq}'
+    """
+}
+
+record Sample {
+    id: String
+    fastq: Path
+}
+```
+
+This approach is useful when the record type can be re-used elsewhere in the pipeline.
+
+### Tuple inputs
+
+Inputs with type `Tuple` can declare the name of each tuple component:
+
+```nextflow
+process fastqc {
+    input:
+    (id, fastq): Tuple<String,Path>
+
+    script:
+    """
+    echo 'id: ${id}'
+    echo 'fastq: ${fastq}'
+    """
+}
+```
+
+This pattern is called *tuple destructuring*. Each tuple component is staged into the task the same way as an individual input.
+
+The generic types inside the `Tuple<...>` annotation specify the type of each tuple compomnent and should match the component names. In the above example, `id` has type `String` and `fastq` has type `Path`.
+
+## Stage directives
 
 The `stage:` section defines custom staging behavior using *stage directives*.  It should be specified after the `input:` section. These directives serve the same purpose as input qualifiers such as `env` and `stdin` in the legacy syntax. 
 
+### Environment variables
+
 The `env` directive declares an environment variable in terms of task inputs:
 
 ```nextflow
@@ -106,6 +181,8 @@ process echo_env {
 }
 ```
 
+### Standard input (stdin)
+
 The `stdin` directive defines the standard input of the task script:
 
 ```nextflow
@@ -123,6 +200,8 @@ process cat {
 }
 ```
 
+### Custom file staging
+
 The `stageAs` directive stages an input file (or files) under a custom file pattern:
 
 ```nextflow
@@ -222,6 +301,43 @@ process foo {
 }
 ```
 
+### Structured outputs
+
+Whereas legacy process outputs could only be structured using specific qualifiers like `val` and `tuple`, typed process outputs are regular values.
+
+The `record()` standard library function can be used to create a record:
+
+```nextflow
+process fastqc {
+    input:
+    sample: Record {
+        id: String
+        fastq: Path
+    }
+
+    output:
+    record(id: sample.id, fastqc: file('fastqc_logs'))
+
+    script:
+    // ...
+}
+```
+
+The `tuple()` standard library function can be used to create a tuple:
+
+```nextflow
+process fastqc {
+    input:
+    (id, fastq): Tuple<String,Path>
+
+    output:
+    tuple(id, file('fastqc_logs'))
+
+    script:
+    // ...
+}
+```
+
 ## Topics
 
 The `topic:` section emits values to {ref}`topic channels <channel-topic>`. A topic emission consists of an output value and a topic name:

docs/process-typed.md

@@ -43,7 +43,7 @@ The global namespace contains globally available constants and functions.
 : Create a branch criteria to use with the {ref}`operator-branch` operator.
 
 `env( name: String ) -> String`
-: :::{versionadded} 24.11.0-edge
+: :::{versionadded} 25.04.0
   :::
 : Get the value of the environment variable with the specified name in the Nextflow launch environment.
 
@@ -108,8 +108,11 @@ The global namespace contains globally available constants and functions.
 `sleep( milliseconds: long )`
 : Sleep for the given number of milliseconds.
 
+`record( [options] ) -> Record`
+: Create a record from the given named arguments.
+
 `tuple( args... ) -> Tuple`
-: Create a tuple object from the given arguments.
+: Create a tuple from the given arguments.
 
 (stdlib-namespaces-channel)=
 

docs/reference/stdlib-namespaces.md

@@ -768,6 +768,35 @@ The following methods are available for splitting and counting the records in fi
 `splitText() -> List<String>`
 : Splits a text file into a list of lines. See the {ref}`operator-splittext` operator for available options.
 
+(stdlib-types-record)=
+
+## Record
+
+A record is an immutable map of fields to values (i.e., `Map<String,?>`). Each value can have its own type. 
+
+A record can be created using the `record` function:
+
+```nextflow
+sample = record(id: '1', fastq_1: file('1_1.fastq'), fastq_2: file('1_2.fastq'))
+```
+
+Record fields can be accessed as properties:
+
+```nextflow
+sample.id
+// -> '1'
+```
+
+The following operations are supported for records:
+
+`+ : (Record, Record) -> Record`
+: Given two records, returns a new record containing the fields and values of both records. When a field is present in both records, the value of the right-hand record takes precedence.
+
+The following methods are available for a record:
+
+`subMap( keys: Iterable<String> ) -> Record`
+: Returns a new record containing only the given fields.
+
 (stdlib-types-set)=
 
 ## Set\<E\>

docs/reference/stdlib-types.md

@@ -34,6 +34,7 @@ A Nextflow script may contain the following top-level declarations:
 - Process definitions
 - Function definitions
 - Enum types
+- Record types
 - Output block
 
 Script declarations are in turn composed of statements and expressions.
@@ -107,6 +108,8 @@ The following definitions can be included:
 - Functions
 - Processes
 - Named workflows
+- *New in 26.04:* Enum types
+- *New in 26.04:* Record types
 
 ### Params block
 
@@ -360,9 +363,17 @@ enum Day {
 
 Enum values in the above example can be accessed as `Day.MONDAY`, `Day.TUESDAY`, and so on.
 
-:::{note}
-Enum types cannot be included across modules at this time.
-:::
+### Record type
+
+A record type declaration consists of a name and a body. The body consists of one or more fields, where each field has a name and a type:
+
+```nextflow
+record FastqPair {
+    id: String
+    fastq_1: Path
+    fastq_2: Path
+}
+```
 
 ### Output block
 

docs/reference/syntax.md

@@ -111,28 +111,58 @@ Copying a map with the `+` operator is a safer way to modify maps in Nextflow, s
 
 See {ref}`stdlib-types-map` for the set of available map operations.
 
+(script-records)=
+
+## Records
+
+Records are used to store a set of related fields, where each field can have its own type. They are created using the `record` function:
+
+```nextflow
+person = record(name: 'Alice', age: 42, is_alive: true)
+```
+
+Record fields are accessed by name:
+
+```nextflow
+name = person.name
+age = person.age
+is_alive = person.is_alive
+```
+
+Records are immutable -- once a record is created, it cannot be modified. Use record operations to create new records instead. 
+
+For example:
+
+```nextflow
+person + record(age: 43) - ['is_alive']
+
+// record(name: 'Alice', age: 43)
+```
+
+See {ref}`stdlib-types-record` for the set of available record operations.
+
 (script-tuples)=
 
 ## Tuples
 
 Tuples are used to store a fixed sequence of heterogeneous values. They are created using the `tuple` function:
 
 ```nextflow
-person = tuple('Alice', 42, false)
+person = tuple('Alice', 42, true)
 ```
 
 Tuple elements are accessed by index:
 
 ```nextflow
 name = person[0]
 age = person[1]
-is_male = person[2]
+is_alive = person[2]
 ```
 
 Tuples can be destructured in assignments:
 
 ```nextflow
-(name, age, is_male) = person
+(name, age, is_alive) = person
 ```
 
 As well as closure parameters:

docs/script.md

@@ -50,14 +50,29 @@ def json = new groovy.json.JsonSlurper().parseText(json_file.text)
 
 Some users use classes in Nextflow to define helper functions or custom types. Helper functions should be defined as standalone functions in Nextflow. Custom types should be moved to the `lib` directory.
 
-:::{note}
-Enums, a special type of class, are supported, but they cannot be included across modules at this time.
-:::
+You can use an enum type to model a choice between a fixed set of categories:
 
-:::{note}
-Record types will be addressed in a future version of the Nextflow language specification.
+```nextflow
+enum Color {
+    RED,
+    GREEN,
+    BLUE
+}
+```
+
+:::{versionadded} 26.04.0
 :::
 
+You can use a record type to model a composition of multiple values:
+
+```nextflow
+record FastqPair {
+    id: String
+    fastq_1: Path
+    fastq_2: Path
+}
+```
+
 ### Mixing script declarations and statements
 
 In the strict syntax, a script may contain any of the following top-level declarations:
@@ -230,13 +245,14 @@ In the strict syntax, use `System.getenv()` instead:
 println "PWD = ${System.getenv('PWD')}"
 ```
 
-:::{versionadded} 24.04.0
-The `env()` function should be used instead of `System.getenv()`:
+:::{versionadded} 25.04.0
+:::
+
+Use the `env()` function instead of `System.getenv()`:
 
 ```nextflow
 println "PWD = ${env('PWD')}"
 ```
-:::
 
 ## Restricted syntax