Skip to content

Commit 01c777a

Browse files
pditommasopditommaso
committed
Add structured tool arguments to module spec schema
- Add `args` property to tools section for type-safe argument configuration - Define toolArgSpec with flag, type, description, default, enum, required - Support implicit variable `tools.<toolname>.args.<argname>` returning formatted flag+value (e.g., "-K 100000000") - Support `tools.<toolname>.args` to return all args concatenated - Document deprecation of ext.args/ext.args2/ext.args3 pattern - Update ADR with Tool Arguments Configuration section and appendix [ci skip] Signed-off-by: Paolo Di Tommaso <paolo.ditommaso@gmail.com>
1 parent 5df12cd commit 01c777a

2 files changed

Lines changed: 214 additions & 14 deletions

File tree

‎adr/20251114-module-system.md‎

Lines changed: 142 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,22 @@
22

33
- Authors: Paolo Di Tommaso
44
- Status: draft
5-
- Date: 2025-12-11
5+
- Date: 2025-01-06
66
- Tags: modules, dsl, registry, versioning, architecture
7-
- Version: 2.1
7+
- Version: 2.2
8+
9+
## Updates
10+
11+
### Version 2.2 (2025-01-06)
12+
- **Structured tool arguments**: Added `args` property to `tools` section for type-safe argument configuration
13+
- **New implicit variables**: `tools.<toolname>.args.<argname>` returns formatted flag+value; `tools.<toolname>.args` returns all args concatenated
14+
- **Deprecation**: `ext.args`, `ext.args2`, `ext.args3` pattern deprecated in favor of structured tool arguments
15+
16+
### Version 2.1 (2025-12-11)
17+
- **Unified dependencies**: Consolidated `components`, `dependencies`, and `requires` into single `requires` field
18+
- **New sub-properties**: `requires.modules` and `requires.workflows` for declaring module dependencies
19+
- **Unified version syntax**: `[scope/]name[@constraint]` format across plugins, modules, and workflows
20+
- **Deprecation**: `components` field deprecated (use `requires.modules` instead)
821

922
## Context and Problem Statement
1023

@@ -471,6 +484,99 @@ project-root/
471484
- Single authentication system
472485
- Separate cache locations: `$NXF_HOME/plugins/` (global) vs `modules/` (per-project)
473486

487+
## Tool Arguments Configuration
488+
489+
The module system introduces a structured approach to tool argument configuration, replacing the legacy `ext.args` pattern with type-safe, documented argument specifications.
490+
491+
### Current Pattern (Deprecated)
492+
493+
The traditional nf-core pattern uses `ext.args` strings in config files:
494+
495+
```groovy
496+
// Config file
497+
withName: 'BWA_MEM' {
498+
ext.args = "-K 100000000 -Y -B 3 -R ${meta.read_group}"
499+
ext.args2 = "--output-fmt cram"
500+
}
501+
502+
// Module script
503+
def args = task.ext.args ?: ''
504+
def args2 = task.ext.args2 ?: ''
505+
bwa mem $args -t $task.cpus $index $reads | samtools sort $args2 -o out.bam -
506+
```
507+
508+
**Limitations:**
509+
- No documentation of available arguments
510+
- No validation or type checking
511+
- Unclear which `ext.argsN` maps to which tool
512+
- No IDE autocompletion support
513+
514+
### New Pattern: Structured Tool Arguments
515+
516+
Modules declare available arguments in `meta.yaml` under each tool's `args` property:
517+
518+
```yaml
519+
tools:
520+
- bwa:
521+
description: BWA aligner
522+
homepage: http://bio-bwa.sourceforge.net/
523+
args:
524+
K:
525+
flag: "-K"
526+
type: integer
527+
description: "Process INT input bases in each batch"
528+
Y:
529+
flag: "-Y"
530+
type: boolean
531+
description: "Use soft clipping for supplementary alignments"
532+
533+
- samtools:
534+
description: SAMtools
535+
homepage: http://www.htslib.org/
536+
args:
537+
output_fmt:
538+
flag: "--output-fmt"
539+
type: string
540+
enum: ["sam", "bam", "cram"]
541+
description: "Output format"
542+
```
543+
544+
### Configuration Usage
545+
546+
Arguments are configured using `tools.<toolname>.args.<argname>`:
547+
548+
```groovy
549+
withName: 'BWA_MEM' {
550+
tools.bwa.args.K = 100000000
551+
tools.bwa.args.Y = true
552+
tools.samtools.args.output_fmt = "cram"
553+
}
554+
```
555+
556+
### Script Usage
557+
558+
In module scripts, access arguments via the `tools` implicit variable:
559+
560+
```groovy
561+
// tools.bwa.args.K → "-K 100000000"
562+
// tools.bwa.args.Y → "-Y"
563+
// tools.bwa.args → "-K 100000000 -Y" (all args concatenated)
564+
565+
bwa mem ${tools.bwa.args} -t $task.cpus $index $reads \
566+
| samtools sort ${tools.samtools.args} -o ${prefix}.bam -
567+
```
568+
569+
### Benefits
570+
571+
| Aspect | `ext.args` (Legacy) | `tools.*.args` (New) |
572+
|--------|---------------------|----------------------|
573+
| Documentation | None | In meta.yaml |
574+
| Type Safety | None | Validated |
575+
| IDE Support | None | Autocompletion |
576+
| Multi-tool | Confusing (`ext.args2`) | Clear (`tools.samtools.args`) |
577+
| Defaults | Manual | Schema-defined |
578+
| Enums | None | Validated |
579+
474580
## Comparison: Plugins vs. Modules
475581

476582
| Aspect | Plugins | Modules |
@@ -691,20 +797,23 @@ requires:
691797

692798
#### `tools`
693799

694-
Documents the software tools wrapped by the module:
800+
Documents the software tools wrapped by the module, including their command-line arguments:
695801

696802
```yaml
697803
tools:
698804
- bwa:
699-
description: |
700-
BWA is a software package for mapping DNA sequences
701-
against a large reference genome.
805+
description: BWA aligner
702806
homepage: http://bio-bwa.sourceforge.net/
703-
documentation: https://bio-bwa.sourceforge.net/bwa.shtml
704-
doi: 10.1093/bioinformatics/btp324
705-
arxiv: arXiv:1303.3997
706807
licence: ["GPL-3.0-or-later"]
707-
identifier: biotools:bwa
808+
args:
809+
K:
810+
flag: "-K"
811+
type: integer
812+
description: "Process INT input bases in each batch"
813+
Y:
814+
flag: "-Y"
815+
type: boolean
816+
description: "Use soft clipping for supplementary alignments"
708817
```
709818

710819
**Tool Properties:**
@@ -720,6 +829,29 @@ tools:
720829
| `licence` | Recommended | SPDX license(s) |
721830
| `identifier` | Recommended | bio.tools identifier |
722831
| `manual` | No | User manual URL |
832+
| `args` | No | Command-line argument specifications |
833+
834+
**Argument Properties (`args.<name>`):**
835+
836+
The `args` object maps argument names to their specifications. Argument names become accessible in scripts via `tools.<toolname>.args.<argname>`.
837+
838+
| Property | Required | Description |
839+
|----------|----------|-------------|
840+
| `flag` | Yes | CLI flag (e.g., `-K`, `--output-fmt`) |
841+
| `type` | Yes | Data type: `boolean`, `integer`, `float`, `string`, `file`, `path` |
842+
| `description` | Yes | Human-readable description |
843+
| `default` | No | Default value |
844+
| `enum` | No | List of allowed values |
845+
| `required` | No | Whether the argument is mandatory (default: false) |
846+
847+
**Argument Type Behavior:**
848+
849+
| Type | Config Example | Output |
850+
|------|----------------|--------|
851+
| `boolean` | `tools.bwa.args.Y = true` | `-Y` |
852+
| `integer` | `tools.bwa.args.K = 100000` | `-K 100000` |
853+
| `string` | `tools.bwa.args.R = "@RG\tID:s1"` | `-R @RG\tID:s1` |
854+
| `string` + `enum` | `tools.samtools.args.output_fmt = "cram"` | `--output-fmt cram` |
723855

724856
#### `input` and `output`
725857

‎adr/module-spec-schema.json‎

Lines changed: 72 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -232,6 +232,16 @@
232232
"type": "string",
233233
"format": "uri",
234234
"description": "Manual/user guide URL"
235+
},
236+
"args": {
237+
"type": "object",
238+
"description": "Command-line arguments supported by the tool. Keys are argument names accessible via tool.<toolname>.args.<name>",
239+
"patternProperties": {
240+
"^[a-zA-Z_][a-zA-Z0-9_]*$": {
241+
"$ref": "#/$defs/toolArgSpec"
242+
}
243+
},
244+
"additionalProperties": false
235245
}
236246
},
237247
"required": ["description"],
@@ -242,6 +252,40 @@
242252
{ "required": ["doi"] }
243253
]
244254
},
255+
"toolArgSpec": {
256+
"type": "object",
257+
"description": "Specification for a tool command-line argument",
258+
"properties": {
259+
"flag": {
260+
"type": "string",
261+
"description": "The CLI flag (e.g., '-n', '--output-fmt')",
262+
"pattern": "^--?[a-zA-Z][a-zA-Z0-9_-]*$"
263+
},
264+
"type": {
265+
"type": "string",
266+
"description": "Data type of the argument value",
267+
"enum": ["boolean", "integer", "float", "string", "file", "path"]
268+
},
269+
"description": {
270+
"type": "string",
271+
"description": "Human-readable description of the argument"
272+
},
273+
"default": {
274+
"description": "Default value for the argument"
275+
},
276+
"enum": {
277+
"type": "array",
278+
"description": "List of allowed values",
279+
"uniqueItems": true
280+
},
281+
"required": {
282+
"type": "boolean",
283+
"description": "Whether this argument is required",
284+
"default": false
285+
}
286+
},
287+
"required": ["flag", "type", "description"]
288+
},
245289
"channelElementSpec": {
246290
"type": "object",
247291
"description": "Specification for a channel element (input or output)",
@@ -452,12 +496,36 @@
452496
"tools": [
453497
{
454498
"bwa": {
455-
"description": "BWA is a software package for mapping DNA sequences against a large reference genome.",
499+
"description": "BWA aligner",
456500
"homepage": "http://bio-bwa.sourceforge.net/",
457-
"documentation": "https://bio-bwa.sourceforge.net/bwa.shtml",
458-
"doi": "10.1093/bioinformatics/btp324",
459501
"licence": ["GPL-3.0-or-later"],
460-
"identifier": "biotools:bwa"
502+
"args": {
503+
"K": {
504+
"flag": "-K",
505+
"type": "integer",
506+
"description": "Process INT input bases in each batch"
507+
},
508+
"Y": {
509+
"flag": "-Y",
510+
"type": "boolean",
511+
"description": "Use soft clipping for supplementary alignments"
512+
}
513+
}
514+
}
515+
},
516+
{
517+
"samtools": {
518+
"description": "SAMtools",
519+
"homepage": "http://www.htslib.org/",
520+
"licence": ["MIT"],
521+
"args": {
522+
"output_fmt": {
523+
"flag": "--output-fmt",
524+
"type": "string",
525+
"description": "Output format",
526+
"enum": ["sam", "bam", "cram"]
527+
}
528+
}
461529
}
462530
}
463531
],

0 commit comments

Comments
 (0)
⚡