Second attempt at updating all modules by maxulysse · Pull Request #455 · nf-core/sarek

maxulysse · 2021-11-22T13:35:25Z

Since I'm going crazy with #440
This is another attempt at it starting from scratch with tiny steps to figure out when and where and what and maybe why it is failing.

github-actions · 2021-11-22T13:50:14Z

`nf-core lint` overall result: Passed ✅ ⚠️

Posted for pipeline commit 76df2dc

+| ✅ 133 tests passed       |+
#| ❔   5 tests were ignored |#
!| ❗   5 tests had warnings |!

Details

❗ Test warnings:

pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed

❔ Tests ignored:

files_unchanged - File ignored due to lint config: .github/workflows/linting.yml
files_unchanged - File ignored due to lint config: assets/nf-core-sarek_logo.png
files_unchanged - File ignored due to lint config: docs/images/nf-core-sarek_logo.png
files_unchanged - File ignored due to lint config: lib/NfcoreTemplate.groovy
files_unchanged - File ignored due to lint config: assets/multiqc_config.yaml

✅ Tests passed:

files_exist - File found: .gitattributes
files_exist - File found: .gitignore
files_exist - File found: .markdownlint.yml
files_exist - File found: CHANGELOG.md
files_exist - File found: CITATIONS.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: CODE_OF_CONDUCT.md
files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
files_exist - File found: nextflow_schema.json
files_exist - File found: nextflow.config
files_exist - File found: README.md
files_exist - File found: .github/.dockstore.yml
files_exist - File found: .github/CONTRIBUTING.md
files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
files_exist - File found: .github/workflows/branch.yml
files_exist - File found: .github/workflows/ci.yml
files_exist - File found: .github/workflows/linting_comment.yml
files_exist - File found: .github/workflows/linting.yml
files_exist - File found: assets/email_template.html
files_exist - File found: assets/email_template.txt
files_exist - File found: assets/sendmail_template.txt
files_exist - File found: assets/nf-core-sarek_logo.png
files_exist - File found: conf/modules.config
files_exist - File found: conf/test.config
files_exist - File found: conf/test_full.config
files_exist - File found: docs/images/nf-core-sarek_logo.png
files_exist - File found: docs/output.md
files_exist - File found: docs/README.md
files_exist - File found: docs/README.md
files_exist - File found: docs/usage.md
files_exist - File found: lib/nfcore_external_java_deps.jar
files_exist - File found: lib/NfcoreSchema.groovy
files_exist - File found: lib/NfcoreTemplate.groovy
files_exist - File found: lib/Utils.groovy
files_exist - File found: lib/WorkflowMain.groovy
files_exist - File found: main.nf
files_exist - File found: assets/multiqc_config.yaml
files_exist - File found: conf/base.config
files_exist - File found: conf/igenomes.config
files_exist - File found: .github/workflows/awstest.yml
files_exist - File found: .github/workflows/awsfulltest.yml
files_exist - File found: lib/WorkflowSarek.groovy
files_exist - File found: modules.json
files_exist - File not found check: Singularity
files_exist - File not found check: parameters.settings.json
files_exist - File not found check: bin/markdown_to_html.r
files_exist - File not found check: conf/aws.config
files_exist - File not found check: .github/workflows/push_dockerhub.yml
files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
files_exist - File not found check: .travis.yml
nextflow_config - Config variable found: manifest.name
nextflow_config - Config variable found: manifest.nextflowVersion
nextflow_config - Config variable found: manifest.description
nextflow_config - Config variable found: manifest.version
nextflow_config - Config variable found: manifest.homePage
nextflow_config - Config variable found: timeline.enabled
nextflow_config - Config variable found: trace.enabled
nextflow_config - Config variable found: report.enabled
nextflow_config - Config variable found: dag.enabled
nextflow_config - Config variable found: process.cpus
nextflow_config - Config variable found: process.memory
nextflow_config - Config variable found: process.time
nextflow_config - Config variable found: params.outdir
nextflow_config - Config variable found: params.input
nextflow_config - Config variable found: params.show_hidden_params
nextflow_config - Config variable found: params.schema_ignore_params
nextflow_config - Config variable found: manifest.mainScript
nextflow_config - Config variable found: timeline.file
nextflow_config - Config variable found: trace.file
nextflow_config - Config variable found: report.file
nextflow_config - Config variable found: dag.file
nextflow_config - Config variable (correctly) not found: params.version
nextflow_config - Config variable (correctly) not found: params.nf_required_version
nextflow_config - Config variable (correctly) not found: params.container
nextflow_config - Config variable (correctly) not found: params.singleEnd
nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
nextflow_config - Config variable (correctly) not found: params.name
nextflow_config - Config timeline.enabled had correct value: true
nextflow_config - Config report.enabled had correct value: true
nextflow_config - Config trace.enabled had correct value: true
nextflow_config - Config dag.enabled had correct value: true
nextflow_config - Config manifest.name began with nf-core/
nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
nextflow_config - Config dag.file ended with .svg
nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
nextflow_config - Config manifest.version ends in dev: '3.0dev'
nextflow_config - Config params.custom_config_version is set to master
nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
nextflow_config - Lines for loading custom profiles found
files_unchanged - .gitattributes matches the template
files_unchanged - .markdownlint.yml matches the template
files_unchanged - CODE_OF_CONDUCT.md matches the template
files_unchanged - LICENSE matches the template
files_unchanged - .github/.dockstore.yml matches the template
files_unchanged - .github/CONTRIBUTING.md matches the template
files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
files_unchanged - .github/workflows/branch.yml matches the template
files_unchanged - .github/workflows/linting_comment.yml matches the template
files_unchanged - assets/email_template.html matches the template
files_unchanged - assets/email_template.txt matches the template
files_unchanged - assets/sendmail_template.txt matches the template
files_unchanged - docs/README.md matches the template
files_unchanged - lib/nfcore_external_java_deps.jar matches the template
files_unchanged - lib/NfcoreSchema.groovy matches the template
files_unchanged - .gitignore matches the template
actions_ci - '.github/workflows/ci.yml' is triggered on expected events
actions_ci - '.github/workflows/ci.yml' checks minimum NF version
actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
readme - README Nextflow minimum version badge matched config. Badge: 21.04.0, Config: 21.04.0
readme - README Nextflow minimum version in Quick Start section matched config. README: 21.04.0, Config: 21.04.0
pipeline_name_conventions - Name adheres to nf-core convention
template_strings - Did not find any Jinja template strings (291 files)
schema_lint - Schema lint passed
schema_lint - Schema title + description lint passed
schema_params - Schema matched params returned from nextflow config
actions_schema_validation - Workflow validation passed: branch.yml
actions_schema_validation - Workflow validation passed: local_modules.yml
actions_schema_validation - Workflow validation passed: ci.yml
actions_schema_validation - Workflow validation passed: linting_comment.yml
actions_schema_validation - Workflow validation passed: linting.yml
actions_schema_validation - Workflow validation passed: awstest.yml
actions_schema_validation - Workflow validation passed: awsfulltest.yml
merge_markers - No merge markers found in pipeline files
modules_json - Only installed modules found in modules.json

Run details

nf-core/tools version 2.2.dev0
Run at 2021-11-24 16:44:47

maxulysse · 2021-11-23T13:57:26Z

        run: |
          python -m pip install --upgrade pip
-          pip install nf-core
+          pip install nf-core git+https://github.com/nf-core/tools.git@dev


Just adding that to have the linting from dev

makes sense

not sure that @drpatelh would approve

maxulysse · 2021-11-23T13:59:07Z

+        params.input               = "${baseDir}/tests/csv/3.0/fastq_pair.csv"
    }
    prepare_recalibration {
        params.input               = 'https://raw.githubusercontent.com/nf-core/test-datasets/sarek/testdata/csv/tiny-mapped-normal-https.csv'


Need to update for the tests

FriederikeHanssen

Amazing ❤️

FriederikeHanssen · 2021-11-24T16:56:08Z

        run: |
          python -m pip install --upgrade pip
-          pip install nf-core
+          pip install nf-core git+https://github.com/nf-core/tools.git@dev


makes sense

maxulysse added 2 commits November 22, 2021 14:33

feat: update test data

76c0d89

fix: using tools dev for linting

dace03e

maxulysse changed the title ~~feat: update test data~~ Second attempt at updating all modules Nov 22, 2021

maxulysse added 3 commits November 22, 2021 14:46

fix: linting

badee21

fix lint

de70e68

sync template for lint

206c3f5

maxulysse added 3 commits November 22, 2021 15:17

fix: update tests

72e31df

fix tests

f84a294

remove part of tests for now

f8bfab0

maxulysse commented Nov 23, 2021

View reviewed changes

Comment thread tests/test_save_bam_mapped.yml Outdated

maxulysse added 17 commits November 23, 2021 15:40

fix: save bam and bai

4302f13

feat: update markduplicates

0024e06

feat: update modules bwa/index

67751f6

feat: update modules bwamem2/index

db42a4d

feat: update modules gatk4/createsequencedictionary

76f575a

feat: update more modules

b91d9af

fix: update syntax for samtools/merge

a105909

fix: update output for annotation subworkflows

8d37904

feat: update subworkflows

9aec3ff

Merge branch 'dev' into dev_tools_update_2

109f472

fix: input files order more consistent

df8fa7e

install modules qualimap/bamqc

e750ec2

feat: install modules samtools/stats

ad9683b

update samtools/view

8f65bc5

feat: update samtools/index

da8453e

chores: update local modules

1602c68

fix: update output for versions

b70a44f

maxulysse added 4 commits November 24, 2021 17:21

chores: updates more subworkflows

16ccc5c

chores: update more modules

a109e03

fix: nf-core lint + EClint

b36aae7

fix: output

76df2dc

maxulysse added the Ready for Review label Nov 24, 2021

maxulysse requested a review from FriederikeHanssen November 24, 2021 16:51

FriederikeHanssen approved these changes Nov 24, 2021

View reviewed changes

maxulysse merged commit d01122b into nf-core:dev Nov 25, 2021

maxulysse deleted the dev_tools_update_2 branch November 25, 2021 09:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Second attempt at updating all modules#455

Second attempt at updating all modules#455
maxulysse merged 29 commits intonf-core:devfrom
maxulysse:dev_tools_update_2

maxulysse commented Nov 22, 2021

Uh oh!

github-actions Bot commented Nov 22, 2021 •

edited

Loading

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

Uh oh!

maxulysse Nov 23, 2021

Uh oh!

FriederikeHanssen Nov 24, 2021

Uh oh!

maxulysse Nov 25, 2021

Uh oh!

maxulysse Nov 23, 2021

Uh oh!

Uh oh!

FriederikeHanssen left a comment

Uh oh!

FriederikeHanssen Nov 24, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

maxulysse commented Nov 22, 2021

Uh oh!

github-actions Bot commented Nov 22, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

nf-core lint overall result: Passed ✅ ⚠️

❗ Test warnings:

❔ Tests ignored:

✅ Tests passed:

Run details

Uh oh!

maxulysse Nov 23, 2021

Choose a reason for hiding this comment

Uh oh!

FriederikeHanssen Nov 24, 2021

Choose a reason for hiding this comment

Uh oh!

maxulysse Nov 25, 2021

Choose a reason for hiding this comment

Uh oh!

maxulysse Nov 23, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

FriederikeHanssen left a comment

Choose a reason for hiding this comment

Uh oh!

FriederikeHanssen Nov 24, 2021

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions Bot commented Nov 22, 2021 •

edited

Loading

`nf-core lint` overall result: Passed ✅ ⚠️