Dev -> Main `v4.0.0` by sateeshperi · Pull Request #541 · nf-core/methylseq

sateeshperi · 2025-06-30T11:12:53Z

nf-core/methylseq v4.0.0 Release Notes

This release introduces major new features, significant refactoring, and extensive test suite improvements. Key highlights include the addition of a targeted sequencing analysis workflow, substantial refactoring of input handling and subworkflows for better modularity and robustness, and a complete overhaul of the testing framework.

Bug Fixes & Refactoring

Core Refactoring (#539):
- The FASTA index subworkflow was refactored to remove implicit parameter passing.
- Input handling has been modernized to use channels instead of direct file paths for increased robustness.
- Channel handling for FASTA files has been improved with better branching logic for gzipped vs. unzipped files.
- Multiple nf-core modules have been updated to their latest versions.
- Fixed workflow versions collection and improved conditional logic for FastQC and MultiQC execution.
Parabricks Fixes:
- The minimum memory for PARABRICKS_FQ2BAMMETH has been increased to 100GB as suggested by the documentation (#528).
- Fixed a parabricks/fq2bammeth failure on AWS Batch-like executors where symlinks don't exist (#536).
General:
- Reverted PR Fix faix notprovided error #523 which involved direct changes to subworkflows in the pipeline.
- The pipeline template has been updated to nf-core/tools v3.2.0 (#504).

Pipeline Updates

Targeted Sequencing (#516, #539):
- A new --run_targeted_sequencing flag has been added to enable further analysis of targeted methylation experiments.
- A new Targeted Sequencing subworkflow has been introduced with improved parameter handling and region-specific analysis capabilities.
- The bedtools/intersect and picard/collecthsmetrics modules from nf-core/modules have been installed.
- A new --collecthsmetrics parameter has been added to allow the collection of Picard HS metrics.
Workflow & Parameter Changes:
- Bwameth:
  - Added --all_contexts and --merge_context parameters (#516).
  - Removed the --comprehensive parameter (#516).
- Bismark:
  - Decoupled --merge_non_CpG from --comprehensive in the Bismark workflow (#516).
- Trimming:
  - Added a new skip_trimming_presets flag to overwrite any preset trimming options (#506).
  - Updated --pbat trimming options to 8|8|8|8 (#506).
  - Removed --cegx and --epignome preset trimming options as the kits are discontinued (#506).
- GPU Usage:
  - Removed the --use_gpu parameter. Activating the gpu profile with --profile gpu is now the standard way to enable GPU-based pathways (#520).
- Quality Control:
  - Added --skip_fastqc and --skip_multiqc parameters to allow skipping quality control steps (#539).

Testing Updates (#539)

Test Suite Consolidation:
- Removed 19 individual test files, replacing them with comprehensive test suites.
New Test Suites:
- bismark_variants.nf.test for comprehensive Bismark testing.
- bismark_hisat_variants.nf.test for Bismark with HISAT2 aligner.
- bwameth_variants.nf.test for comprehensive Bwameth testing.
- targeted_sequencing_variants.nf.test for the new targeted sequencing workflow.
- index_downloads.nf.test for testing index download scenarios.

post release bump version

Update test command in usage.md

…hylseq into dev

…thylseq into 511-targeted-analysis

Update snapshot 3.2.0 template

…/methylseq into 511-targeted-analysis

…ove unnecessary use of the first operator, improving clarity and consistency in the workflow setup.

refactor-workflow-tests

bump-version 4.0.0

github-actions · 2025-06-30T11:14:45Z

`nf-core pipelines lint` overall result: Passed ✅

Posted for pipeline commit f293f94

+| ✅ 238 tests passed       |+
#| ❔   4 tests were ignored |#

Details

❔ Tests ignored:

files_exist - File is ignored: lib/nfcore_external_java_deps.jar
files_exist - File is ignored: conf/modules.config
files_exist - File is ignored: .github/workflows/ci.yml
modules_config - modules_config

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .nf-core.yml
files_exist - File found: .prettierignore
files_exist - File found: .prettierrc.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/nf-test.yml
files_exist - File found: .github/actions/get-shards/action.yml
files_exist - File found: .github/actions/nf-test/action.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-methylseq_logo_light.png
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-methylseq_logo_light.png
files_exist - File found: docs/images/nf-core-methylseq_logo_dark.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: nf-test.config
files_exist - File found: tests/default.nf.test
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: conf/igenomes_ignored.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: modules.json
files_exist - File found: ro-crate-metadata.json
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .markdownlint.yml
files_exist - File not found check: .nf-core.yaml
files_exist - File not found check: .yamllint.yml
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: docs/images/nf-core-methylseq_logo.png
files_exist - File not found check: lib/Checks.groovy
files_exist - File not found check: lib/Completion.groovy
files_exist - File not found check: lib/NfcoreTemplate.groovy
files_exist - File not found check: lib/Utils.groovy
files_exist - File not found check: lib/Workflow.groovy
files_exist - File not found check: lib/WorkflowMain.groovy
files_exist - File not found check: lib/WorkflowMethylseq.groovy
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: pipeline_template.yml
files_exist - File not found check: Singularity
files_exist - File not found check: .travis.yml
nextflow_config - Found nf-schema plugin
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: validation.help.enabled
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable found: validation.help.beforeText
nextflow_config - Config variable found: validation.help.afterText
nextflow_config - Config variable found: validation.help.command
nextflow_config - Config variable found: validation.summary.beforeText
nextflow_config - Config variable found: validation.summary.afterText
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config variable (correctly) not found: params.enable_conda
nextflow_config - Config variable (correctly) not found: params.max_cpus
nextflow_config - Config variable (correctly) not found: params.max_memory
nextflow_config - Config variable (correctly) not found: params.max_time
nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
nextflow_config - Config variable (correctly) not found: params.validationLenientMode
nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .html
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version does not contain dev for release: 4.0.0
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
nextflow_config - nextflow.config contains configuration profile test
nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
nextflow_config - Config default value correct: params.aligner= bismark
nextflow_config - Config default value correct: params.clip_r1= 0
nextflow_config - Config default value correct: params.clip_r2= 0
nextflow_config - Config default value correct: params.three_prime_clip_r1= 0
nextflow_config - Config default value correct: params.three_prime_clip_r2= 0
nextflow_config - Config default value correct: params.nextseq_trim= 0
nextflow_config - Config default value correct: params.skip_trimming_presets= false
nextflow_config - Config default value correct: params.num_mismatches= 0.6
nextflow_config - Config default value correct: params.no_overlap= true
nextflow_config - Config default value correct: params.ignore_r1= 0
nextflow_config - Config default value correct: params.ignore_r2= 2
nextflow_config - Config default value correct: params.ignore_3prime_r1= 0
nextflow_config - Config default value correct: params.ignore_3prime_r2= 2
nextflow_config - Config default value correct: params.min_depth= 0
nextflow_config - Config default value correct: params.custom_config_version= master
nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Config default value correct: params.publish_dir_mode= copy
nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
nextflow_config - Config default value correct: params.validate_params= true
nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/methylseq/
nf_test_content - 'tests/targeted_sequencing_variants.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/targeted_sequencing_variants.nf.test' contains outdir parameter
nf_test_content - 'tests/bismark_hisat_variants.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/bismark_hisat_variants.nf.test' contains outdir parameter
nf_test_content - 'tests/bwameth_variants.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/bwameth_variants.nf.test' contains outdir parameter
nf_test_content - 'tests/bismark_variants.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/bismark_variants.nf.test' contains outdir parameter
nf_test_content - 'tests/index_downloads.nf.test' contains outdir parameter
nf_test_content - 'tests/index_downloads.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/bwameth_use_gpu.nf.test' contains outdir parameter
nf_test_content - 'tests/bwameth_use_gpu.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' contains outdir parameter
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' snapshots a 'versions.yml' file
nf_test_content - 'tests/default.nf.test' contains outdir parameter
nf_test_content - 'tests/nextflow.config' contains modules_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'tests/nextflow.config' contains pipelines_testdata_base_path
nf_test_content - 'nf-test.config' sets a testsDir
nf_test_content - 'nf-test.config' sets a workDir
nf_test_content - 'nf-test.config' sets a configFile
files_unchanged - .gitattributes matches the template
files_unchanged - .prettierrc.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - .github/workflows/linting.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - assets/nf-core-methylseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-methylseq_logo_light.png matches the template
files_unchanged - docs/images/nf-core-methylseq_logo_dark.png matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - .gitignore matches the template
files_unchanged - .prettierignore matches the template
actions_nf_test - '.github/workflows/nf-test.yml' is triggered on expected events
actions_nf_test - '.github/workflows/nf-test.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 24.10.2, Config: 24.10.2
readme - README nf-core template version badge found.
readme - README Zenodo placeholder was replaced with DOI.
pipeline_todos - No TODO strings found
pipeline_if_empty_null - No ifEmpty(null) strings found
plugin_includes - No wrong validation plugin imports have been found
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (0 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_lint - Input mimetype lint passed: 'text/csv'
schema_params - Schema matched params returned from nextflow config
system_exit - No System.exit calls found
actions_schema_validation - Workflow validation passed: fix_linting.yml
actions_schema_validation - Workflow validation passed: clean-up.yml
actions_schema_validation - Workflow validation passed: download_pipeline.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: nf-test.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: release-announcements.yml
actions_schema_validation - Workflow validation passed: nf-test-gpu.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: template-version-comment.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json
multiqc_config - assets/multiqc_config.yml found and not ignored.
multiqc_config - assets/multiqc_config.yml contains report_section_order
multiqc_config - assets/multiqc_config.yml contains export_plots
multiqc_config - assets/multiqc_config.yml contains report_comment
multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
local_component_structure - local subworkflows directory structure is correct 'subworkflows/local/TOOL/SUBTOOL'
base_config - conf/base.config found and not ignored.
base_config - PRESEQ_LCEXTRAP found in conf/base.config and Nextflow scripts.
base_config - BISMARK_ALIGN found in conf/base.config and Nextflow scripts.
base_config - BISMARK_DEDUPLICATE found in conf/base.config and Nextflow scripts.
base_config - BISMARK_METHYLATIONEXTRACTOR found in conf/base.config and Nextflow scripts.
base_config - BWAMETH_ALIGN found in conf/base.config and Nextflow scripts.
base_config - PARABRICKS_FQ2BAMMETH found in conf/base.config and Nextflow scripts.
nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.3.1
rocrate_readme_sync - RO-Crate descriptions are in sync with README.md.
version_consistency - Version tags are numeric and consistent between container, release tag and config.
included_configs - Pipeline config includes custom configs.

Run details

nf-core/tools version 3.3.1
Run at 2025-07-03 08:55:32

update metro map 4.0.0

dcarrillox

Just minor changes in the documentation

maxulysse

Looking good, just some minor comments

JudithBernett

lgtm!

apply suggestions from code review

sateeshperi and others added 30 commits December 17, 2024 03:23

post release bump version

d786130

Merge pull request #497 from sateeshperi/dev

68b9d40

post release bump version

Update test command in usage.md

bb90034

Merge pull request #502 from nf-core/501-test-command-wrong-in-usagemd

f23814a

Update test command in usage.md

Template update for nf-core/tools version 3.1.2

3acef7f

Template update for nf-core/tools version 3.2.0

a46da70

Merge branch 'nf-core-template-merge-3.2.0' of github.com:nf-core/met…

af7c0f4

…hylseq into dev

update snapshots after 3.2.0 template merge

5406ad8

update changelog after 3.2.0 template-merge

757a954

restore bwameth gpu snap to dev

eae16d4

setup parameters and workflow

2bc87d4

update gunzip module

0232cf9

update snaps after gunzip module update

cb86387

update min nxf ver to 24.10.2

1f161ca

#514 simplify --comprehensive, add --all_contexts

20e99ae

subworkflow for picard collecthsmetrics done, to test

7ec1b70

Merge branch '511-targeted-analysis' of https://github.com/nf-core/me…

ed410c7

…thylseq into 511-targeted-analysis

update gpu snap after gunzip update

c77a7c1

Merge pull request #505 from nf-core/update-snapshot-3.2.0-template

b7e1598

Update snapshot 3.2.0 template

test ok with bismark @luigilamparelli

0fd4980

install bedtools and draft intersect_target

c85b838

add --merge_context for methyldackel

70d36a8

minor fix config

1efe514

move bedtools intersect together with the other nf-core modules

251923e

Merge branch '511-targeted-analysis' of https://github.com/dcarrillox…

c49fb49

…/methylseq into 511-targeted-analysis

refactor target_sequencing subworkflow

8570277

Merge branch '511-targeted-analysis' of https://github.com/dcarrillox…

517741f

…/methylseq into 511-targeted-analysis

remove bedtools import from main

4d12d49

Merge branch 'dev' into 511-targeted-analysis

f5907f8

refactor targeted_sequencing

9f65e27

sateeshperi and others added 7 commits June 30, 2025 08:33

restore docs/README

5c96b1e

update docs/output.md

94cf9b3

Refactor channel initialization for fasta and index parameters to rem…

e3a5119

…ove unnecessary use of the first operator, improving clarity and consistency in the workflow setup.

update fasta-index-bismark-bwameth subwf

c2f1074

Merge pull request #539 from nf-core/update-fasta-index-sub-wf

020dd98

refactor-workflow-tests

update changelog & bump-ver 4.0.0

a576b2a

Merge pull request #540 from sateeshperi/dev

b0d1f5b

bump-version 4.0.0

sateeshperi requested a review from a team as a code owner June 30, 2025 11:12

sateeshperi requested a review from FelixKrueger June 30, 2025 11:12

sateeshperi self-assigned this Jun 30, 2025

This was linked to issues Jun 30, 2025

Add --skip_fastqc option #327

Closed

Addition of --mergeContext for MethylDackel #460

Closed

--targeted analysis #511

Closed

sateeshperi and others added 3 commits June 30, 2025 12:46

update metro map 4.0.0

6e251ba

update ro-crate

056e1d3

Merge pull request #542 from sateeshperi/dev

934d092

update metro map 4.0.0

dcarrillox reviewed Jul 1, 2025

View reviewed changes

Comment thread docs/output.md Outdated

Comment thread docs/output.md Outdated

dcarrillox approved these changes Jul 1, 2025

View reviewed changes