Skip to content

Commit 00824f3

Browse files
committed
Fix meta.yaml -> meta.yml, module manifest -> module spec
Signed-off-by: Ben Sherman <bentshermann@gmail.com>
1 parent 9b647ef commit 00824f3

9 files changed

Lines changed: 95 additions & 95 deletions

File tree

adr/20251114-module-system.md

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -11,13 +11,13 @@
1111
### Version 2.7 (2026-03-09)
1212
- **Renamed `.checksum` to `.module-info`**: Leaves room for additional properties in the future
1313
- **Removed `@` prefix from module scopes**: Local modules are distinguished from remote modules by presence/absence of `./` prefix
14-
- **Removed version pinning from config**: Installed module versions are now inferred from the `meta.yaml` of each module in the `modules/` directory instead of being declared in `nextflow.config`
14+
- **Removed version pinning from config**: Installed module versions are now inferred from the `meta.yml` of each module in the `modules/` directory instead of being declared in `nextflow.config`
1515

1616
### Version 2.6 (2026-01-28)
1717
- **Removed module parameters**: Module parameters specification moved to separate spec document.
1818

1919
### Version 2.5 (2026-01-23)
20-
- **Module parameters**: Replaced structured tool arguments with general module parameters defined in `meta.yaml`
20+
- **Module parameters**: Replaced structured tool arguments with general module parameters defined in `meta.yml`
2121
- **Simplified tools section**: Removed `args` property from tools; tool arguments now configured via module parameters
2222
- **Simplified `requires` block**: Removed `plugins`, `modules`, and `subworkflows` sub-properties; `requires` now only contains `nextflow` version constraint
2323
- **Process modules focus**: Removed sub-workflow references; spec is now focused on process modules only
@@ -74,7 +74,7 @@ include { MY_PROCESS } from './modules/my-process.nf'
7474

7575
**Module Naming**: Scoped modules `scope/name` (e.g., `nf-core/salmon`, `myorg/custom`). Local paths supported for backwards compatibility. No nested paths with the module are allowed - each module must have a `main.nf` as the entry point.
7676

77-
**Version Resolution**: Installed module versions are inferred from the `meta.yaml` of each module in the `modules/` directory. If a module is not present locally, the latest available version is downloaded from the registry.
77+
**Version Resolution**: Installed module versions are inferred from the `meta.yml` of each module in the `modules/` directory. If a module is not present locally, the latest available version is downloaded from the registry.
7878

7979
**Resolution Order**:
8080
1. Check local `modules/scope/name/` exists
@@ -86,7 +86,7 @@ include { MY_PROCESS } from './modules/my-process.nf'
8686
| Local State | Action |
8787
|-------------|--------|
8888
| Missing | Download latest from registry |
89-
| Exists, checksum valid | Use local module (version from `meta.yaml`) |
89+
| Exists, checksum valid | Use local module (version from `meta.yml`) |
9090
| Exists, checksum mismatch | **Warn**: locally modified, will NOT replace unless `-force` is used |
9191

9292
**Key Behaviors**:
@@ -120,7 +120,7 @@ registry {
120120
}
121121
```
122122

123-
**Module Spec** (`meta.yaml`):
123+
**Module Spec** (`meta.yml`):
124124
```yaml
125125
name: nf-core/bwa-align
126126
version: 1.2.4 # This module's version
@@ -158,7 +158,7 @@ This avoids introducing new notation that would require additional parser suppor
158158

159159
**Module Resolution**:
160160

161-
Installed module versions are inferred from the `meta.yaml` file for each module in the `modules/` directory.
161+
Installed module versions are inferred from the `meta.yml` file for each module in the `modules/` directory.
162162

163163
### 3. Unified Nextflow Registry
164164

@@ -190,7 +190,7 @@ Note: The `{name}` parameter includes the namespace prefix (e.g., "nf-core/fastq
190190
191191
**Artifact Types**:
192192
- **Plugins**: JAR files with JSON metadata, resolved at startup
193-
- **Modules**: Source archives (.nf + meta.yaml), resolved at parse time
193+
- **Modules**: Source archives (.nf + meta.yml), resolved at parse time
194194
195195
**Benefits**:
196196
- Reuses existing infrastructure (HTTP service, S3 storage, authentication)
@@ -318,7 +318,7 @@ Display the status of all modules, comparing what is configured in `nextflow.con
318318

319319
**Output columns**:
320320
- Module name (`scope/name`)
321-
- Installed version (from `modules/scope/name/meta.yaml`)
321+
- Installed version (from `modules/scope/name/meta.yml`)
322322
- Latest available version (from registry)
323323
- Status indicator (up-to-date, outdated, missing)
324324

@@ -364,15 +364,15 @@ Publish a module to the Nextflow registry, making it available for others to ins
364364
- `-dry-run`: Validate without publishing
365365

366366
**Behavior**:
367-
1. Validates `meta.yaml` schema and required fields (name, version, description)
367+
1. Validates `meta.yml` schema and required fields (name, version, description)
368368
2. Verifies that `main.nf` exists and is valid Nextflow syntax
369369
3. Verifies that `README.md` documentation is present
370370
4. Authenticates with registry using configured credentials
371371
5. Creates a release draft and uploads the module archive
372372
6. Publishes the release, making it available for installation
373373

374374
**Requirements**:
375-
- Valid `meta.yaml` with name, version, and description
375+
- Valid `meta.yml` with name, version, and description
376376
- `main.nf` entry point file
377377
- `README.md` documentation
378378
- Authentication token configured in `registry.auth` or `NXF_REGISTRY_TOKEN`
@@ -391,12 +391,12 @@ Everything within the module directory should be uploaded. Module bundle should
391391
```
392392
my-module/
393393
├── main.nf # Required: entry point for module
394-
├── meta.yaml # Required: Module spec (version, metadata, I/O specs)
394+
├── meta.yml # Required: Module spec (version, metadata, I/O specs)
395395
├── README.md # Required: Module description
396396
└── tests/ # Optional tests
397397
```
398398

399-
**Module Spec extension** (`meta.yaml`):
399+
**Module Spec extension** (`meta.yml`):
400400
```yaml
401401
name: nf-core/bwa-align
402402
version: 1.2.4 # This module's version
@@ -418,16 +418,16 @@ project-root/
418418
├── nf-core/
419419
│ ├── bwa-align/
420420
│ │ ├── .module-info # Cached registry checksum
421-
│ │ ├── meta.yaml
421+
│ │ ├── meta.yml
422422
│ │ └── main.nf # Required entry point
423423
│ └── samtools/view/
424424
│ ├── .module-info
425-
│ ├── meta.yaml
425+
│ ├── meta.yml
426426
│ └── main.nf # Required entry point
427427
└── myorg/
428428
└── custom-process/
429429
├── .module-info
430-
├── meta.yaml
430+
├── meta.yml
431431
└── main.nf # Required entry point
432432
```
433433

@@ -455,12 +455,12 @@ project-root/
455455
1. Parse `include` statements → extract module names (e.g., `nf-core/bwa-align`)
456456
2. For each module:
457457
a. Check local `modules/scope/name/` exists
458-
- If exists → read installed version from `modules/scope/name/meta.yaml`
458+
- If exists → read installed version from `modules/scope/name/meta.yml`
459459
- If missing → download latest version from registry
460460
b. Verify local module integrity against `.module-info` file
461461
- Checksum mismatch → warn and do NOT override (local changes detected)
462462
3. On download: store module to `modules/scope/name/` with `.module-info` file
463-
4. Read `meta.yaml` file: Validates Nextflow requirement → Fail if not fulfilled
463+
4. Read `meta.yml` file: Validates Nextflow requirement → Fail if not fulfilled
464464
5. Parse module's `main.nf` file → make processes available
465465

466466
**Security**:
@@ -484,7 +484,7 @@ project-root/
484484
| Metadata | JSON spec | YAML spec |
485485
| Naming | `nf-amazon` | `nf-core/salmon` |
486486
| Cache Location | `$NXF_HOME/plugins/` | `modules/scope/name/` |
487-
| Version Config | `plugins {}` in config | `meta.yaml` in `modules/` directory |
487+
| Version Config | `plugins {}` in config | `meta.yml` in `modules/` directory |
488488
| Registry Path | `/api/v1/plugins/` | `/api/modules/{name}` |
489489

490490
## Rationale
@@ -495,9 +495,9 @@ project-root/
495495
- Lower operational overhead
496496
- Type-specific handling maintains separation of concerns
497497

498-
**Why infer versions from `meta.yaml` instead of pinning in a separate file?**
498+
**Why infer versions from `meta.yml` instead of pinning in a separate file?**
499499
- Simple: install a version once and it is captured in the module files
500-
- Reproducibility via committing the `modules/` directory (including `meta.yaml`) to the project git repository
500+
- Reproducibility via committing the `modules/` directory (including `meta.yml`) to the project git repository
501501
- Reduces configuration burden: no need to keep config in sync with installed state
502502

503503
**Why parse-time resolution?**
@@ -520,12 +520,12 @@ project-root/
520520

521521
**Positive**:
522522
- Enables ecosystem-wide code reuse
523-
- Reproducible workflows via committing the `modules/` directory (including `meta.yaml`) to the project git repository
523+
- Reproducible workflows via committing the `modules/` directory (including `meta.yml`) to the project git repository
524524
- Centralized discovery and distribution via unified registry
525525
- Minimal operational overhead (single registry for both plugins and modules)
526526
- Module scoping enables organization namespaces and private registries
527527
- Local `modules/` directory provides project isolation
528-
- No version duplication: installed `meta.yaml` is the single source of truth
528+
- No version duplication: installed `meta.yml` is the single source of truth
529529
- Simple module structure: each module has single `main.nf` entry point
530530

531531
**Negative**:
@@ -548,7 +548,7 @@ project-root/
548548

549549
## Appendix A: Module Schema Specification
550550

551-
This appendix defines the JSON schema for module `meta.yaml` files. The schema maintains backward compatibility with existing nf-core module metadata patterns while supporting the new Nextflow module system features.
551+
This appendix defines the JSON schema for module `meta.yml` files. The schema maintains backward compatibility with existing nf-core module metadata patterns while supporting the new Nextflow module system features.
552552

553553
**Schema File:** [module-spec-schema.json](module-spec-schema.json)
554554
**Published URL:** `https://registry.nextflow.io/schemas/module-spec/v1.0.0`
@@ -761,7 +761,7 @@ output:
761761

762762
#### Schema Validation
763763

764-
Use the schema reference in your `meta.yaml`:
764+
Use the schema reference in your `meta.yml`:
765765

766766
```yaml
767767
# yaml-language-server: $schema=https://registry.nextflow.io/schemas/module-spec/v1.0.0

adr/module-spec-schema.json

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22
"$schema": "https://json-schema.org/draft/2020-12/schema",
33
"$id": "https://raw.githubusercontent.com/nextflow-io/schemas/main/module/v1/schema.json",
44
"title": "Nextflow Module Schema",
5-
"description": "Schema for Nextflow module meta.yaml files, supporting both nf-core community patterns and the Nextflow module system",
5+
"description": "Schema for Nextflow module meta.yml files, supporting both nf-core community patterns and the Nextflow module system",
66
"type": "object",
77
"properties": {
88
"name": {
99
"type": "string",
10-
"description": "Module name. Can be a simple identifier (e.g., 'fastqc', 'bwa_mem') for local/nf-core modules, or a fully qualified scoped name (e.g., 'nf-core/fastqc', 'myorg/custom') for registry modules. Note: The '@' prefix is only used in DSL include statements, not in meta.yaml",
10+
"description": "Module name. Can be a simple identifier (e.g., 'fastqc', 'bwa_mem') for local/nf-core modules, or a fully qualified scoped name (e.g., 'nf-core/fastqc', 'myorg/custom') for registry modules.",
1111
"examples": ["fastqc", "bwa_mem", "nf-core/fastqc", "myorg/salmon-quant"],
1212
"pattern": "^([a-z0-9][a-z0-9-]*/)?[a-z][a-z0-9_-]*$"
1313
},

docs/cli.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -386,7 +386,7 @@ $ nextflow module publish myorg/my-module
386386
$ nextflow module publish myorg/my-module -dry-run
387387
```
388388

389-
Publishing requires authentication via the `NXF_REGISTRY_TOKEN` environment variable or `registry.apiKey` in the Nextflow configuration. The module must include `main.nf`, `meta.yaml`, and `README.md` files.
389+
Publishing requires authentication via the `NXF_REGISTRY_TOKEN` environment variable or `registry.apiKey` in the Nextflow configuration. The module must include `main.nf`, `meta.yml`, and `README.md` files.
390390

391391
Use `-dry-run` to validate your module structure without uploading.
392392

docs/module.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -389,7 +389,7 @@ The argument can be either a `scope/name` reference (for an already-installed mo
389389
Your module directory must include:
390390

391391
- `main.nf`: The module entry point
392-
- `meta.yaml`: Module metadata (name, description, version, etc.)
392+
- `meta.yml`: Module spec (name, description, version, etc.)
393393
- `README.md`: Module documentation
394394

395395
Authentication is required for publishing and can be provided via the `NXF_REGISTRY_TOKEN` environment variable or in your configuration:
@@ -432,8 +432,8 @@ modules/
432432
└── module-name/
433433
├── .module-info # Integrity checksum (generated automatically)
434434
├── README.md # Documentation (required for publishing)
435-
├── main.nf # Module entry point (required)
436-
├── meta.yaml # Module metadata (required for publishing)
435+
├── main.nf # Module script (required)
436+
├── meta.yml # Module spec (required for publishing)
437437
├── resources/ # Optional: module binaries and resources
438438
└── templates/ # Optional: process templates
439439
```

docs/reference/cli.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1308,7 +1308,7 @@ The `module` command provides a comprehensive system for managing reusable, regi
13081308
: Publish a module to the registry, making it available for others to install.
13091309
: The argument can be either a `scope/name` reference (for an already-installed module) or a local directory path containing the module files.
13101310
: Requires authentication via `NXF_REGISTRY_TOKEN` environment variable or `registry.apiKey` configuration.
1311-
: The module directory must contain `main.nf`, `meta.yaml`, and `README.md`.
1311+
: The module directory must contain `main.nf`, `meta.yml`, and `README.md`.
13121312
: The following options are available:
13131313

13141314
`-dry-run`

modules/nextflow/src/main/groovy/nextflow/cli/module/CmdModulePublish.groovy

Lines changed: 24 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -88,21 +88,21 @@ class CmdModulePublish extends CmdBase {
8888
)
8989
}
9090

91-
// Step 2: Load and validate manifest
91+
// Step 2: Load and validate spec
9292
def manifestPath = moduleDir.resolve(ModuleStorage.MODULE_MANIFEST_FILE)
93-
def manifest = ModuleSpec.load(manifestPath)
93+
def spec = ModuleSpec.load(manifestPath)
9494

95-
def manifestErrors = manifest.validate()
95+
def manifestErrors = spec.validate()
9696
if (!manifestErrors.isEmpty()) {
9797
throw new AbortOperationException(
98-
"Module manifest validation failed:\n" + manifestErrors.collect { " - ${it}" }.join('\n')
98+
"Module spec validation failed:\n" + manifestErrors.collect { " - ${it}" }.join('\n')
9999
)
100100
}
101101

102-
log.info "Module validated: ${manifest.name}@${manifest.version}"
102+
log.info "Module validated: ${spec.name}@${spec.version}"
103103

104104
if (dryRun) {
105-
printDryRunInfo(manifest)
105+
printDryRunInfo(spec)
106106
return
107107
}
108108

@@ -114,11 +114,11 @@ class CmdModulePublish extends CmdBase {
114114

115115
def registryConfig = config.navigate('registry') as RegistryConfig ?: new RegistryConfig()
116116

117-
publishModule(moduleDir, registryConfig, manifest)
117+
publishModule(moduleDir, registryConfig, spec)
118118

119119
}
120120

121-
private void publishModule(Path moduleDir, RegistryConfig registryConfig, ModuleSpec manifest){
121+
private void publishModule(Path moduleDir, RegistryConfig registryConfig, ModuleSpec spec){
122122
log.info "Creating module bundle..."
123123
def tempBundleFile = Files.createTempFile("nf-module-publish-", ".tar.gz")
124124

@@ -131,15 +131,15 @@ class CmdModulePublish extends CmdBase {
131131

132132
// Create publish request as a map (npr-api will serialize it)
133133
def request = [
134-
version: manifest.version,
134+
version: spec.version,
135135
bundle: bundleBytes
136136
]
137137

138138
// Publish to registry
139139
final registry = registryUrl ?: registryConfig.url
140140
log.info "Publishing module to registry: ${registryUrl ?: registryConfig.url}"
141141
def registryClient = new ModuleRegistryClient(registryConfig)
142-
def response = registryClient.publishModule(manifest.name, request, registry)
142+
def response = registryClient.publishModule(spec.name, request, registry)
143143

144144
if (useModuleReference) {
145145
// If publish is performed using the module reference we should create/update the .module-info with the correct checksum
@@ -152,13 +152,13 @@ class CmdModulePublish extends CmdBase {
152152
println "✓ Module published successfully!"
153153
println ""
154154
println "Module details:"
155-
println " Name: ${manifest.name}"
156-
println " Version: ${manifest.version}"
155+
println " Name: ${spec.name}"
156+
println " Version: ${spec.version}"
157157
println " DownloadUrl: ${response.downloadUrl}"
158158

159159
println ""
160160
println "Others can now install this module using:"
161-
println " nextflow module install ${manifest.name}"
161+
println " nextflow module install ${spec.name}"
162162

163163
} finally {
164164
// Clean up temporary bundle file
@@ -172,23 +172,23 @@ class CmdModulePublish extends CmdBase {
172172
}
173173
}
174174

175-
private void printDryRunInfo(ModuleSpec manifest) {
175+
private void printDryRunInfo(ModuleSpec spec) {
176176
println "✓ Module structure is valid"
177177
println ""
178178
println "Module details:"
179-
println " Name: ${manifest.name}"
180-
println " Version: ${manifest.version}"
181-
println " Description: ${manifest.description}"
182-
println " License: ${manifest.license}"
183-
if( manifest.authors ) {
184-
println " Authors: ${manifest.authors.join(', ')}"
179+
println " Name: ${spec.name}"
180+
println " Version: ${spec.version}"
181+
println " Description: ${spec.description}"
182+
println " License: ${spec.license}"
183+
if( spec.authors ) {
184+
println " Authors: ${spec.authors.join(', ')}"
185185
}
186-
if( manifest.keywords ) {
187-
println " Keywords: ${manifest.keywords.join(', ')}"
186+
if( spec.keywords ) {
187+
println " Keywords: ${spec.keywords.join(', ')}"
188188
}
189-
if( manifest.requires ) {
189+
if( spec.requires ) {
190190
println " Requires:"
191-
manifest.requires.each { name, version ->
191+
spec.requires.each { name, version ->
192192
println " - ${name}: ${version}"
193193
}
194194
}

0 commit comments

Comments
 (0)