Skip to content

Add TSV input funcionality and extra goodies.#396

Merged
jfy133 merged 156 commits intodevfrom
tsv-input
May 18, 2020
Merged

Add TSV input funcionality and extra goodies.#396
jfy133 merged 156 commits intodevfrom
tsv-input

Conversation

@jfy133
Copy link
Copy Markdown
Member

@jfy133 jfy133 commented May 1, 2020 •
edited
Loading

nf-core/eager pull request

This PR mainly includes everything related to our new input system, which allows either direct paths to FASTQs/BAMS, or alternatively, with a TSV 'map' which contains all the relevent per-library metadata.

Along with this is updated documentation, lots of minor bug fixes including emailing, profile loading etc.

PR checklist

  • This comment contains a description of changes (with reason)
  • If you've fixed a bug or added code that should be tested, add tests!
  • If necessary, also make a PR on the nf-core/eager branch on the nf-core/test-datasets repo
  • Ensure the test suite passes (nextflow run . -profile test,docker --paired_end).
  • Make sure your code lints (nf-core lint .).
  • Documentation in docs is updated
  • CHANGELOG.md is updated
  • README.md is updated

Learn more about contributing: CONTRIBUTING.md

apeltzer and others added 30 commits December 1, 2019 21:52
@apeltzer
Add checks for all possilbe extensions
4cc7e86
@apeltzer
Get rid of former channels
493fbaa
@apeltzer
More changes
0f841be
@apeltzer
Small comments for tomorrow to get into this easier
d854cd0
@apeltzer
Merge remote-tracking branch 'upstream/dev' into TSV
870b9b6
@apeltzer
Merge remote-tracking branch 'upstream/dev' into TSV
30312d3
@apeltzer
Merge branch 'tsv-input' into TSV
89629f1
@apeltzer
Merge pull request #347 from apeltzer/TSV
e8bbe29 
TSV Input branch - please merge
@jfy133
Fixed a lot of missing close brackets, and the TSV map creation (most…
83e5380 
…ly replacing || with ! &&)
@jfy133
Now also properly loading paired end FASTQs
7b2edcd
@jfy133
Standardised map headers and started making BAM preprocessing input c…
c511b6d 
…hannels
@jfy133
Missed one BAM channel
27a534e
@apeltzer
Merge pull request #363 from nf-core/dev
26ae7e9 
Pull request for release of 2.1.0
@jfy133
Add little bit of documentation
96362a7
@jfy133
docs/usage.md
50ffe51
@jfy133
Replace camel for snake
141c9c2
@jfy133
Fix TSV parsing
d3f201e
@jfy133
Updated usage docs
d0a46b8
@jfy133
Start working on FASTQ route
22f5020
@jfy133
Trying to work out how to detect PE/SE data for AR
ae0963e
@jfy133
Drafted post AR tuple input/output
ffa01f3
@jfy133
Fixed bwa bam output
f8b3c52
@jfy133
TSV up to post-mapping flagstat for default run
e6974cd
@jfy133
Added TSV for flagstat
560a3be
@jfy133
Started removing BAI column in TSV
ccf1583
@jfy133
Completed all input/tuple insertions. collect() processes not current…
d6f8804 
…ly working properly
@jfy133
Fix convertBam process, bam only TSV input doesn't activate downstrea…
85cbb75 
…m channels after indexing
@jfy133
Fix bam input and started addressing collect issue
c7b02e7
@jfy133
Fixed malt collect by removing tuple, but sexdet and nuccontam not wo…
99dc4a6 
…rking yet
@jfy133
Partially change github actions tests to a new TSV test config
292a93f
@jfy133 jfy133 marked this pull request as ready for review May 15, 2020 06:10
@jfy133 jfy133 changed the title [DRAFT DO NOT MERGE] Add TSV input funcionality and extra goodies. Add TSV input funcionality and extra goodies. May 15, 2020
@jfy133 jfy133 removed WIP Work in progress do_not_merge labels May 15, 2020
@maxibor
Copy link
Copy Markdown
Member

maxibor commented May 15, 2020

General comment: there are many prefix that you can completely replace by libraryid in many processes.

maxibor
maxibor reviewed May 15, 2020
View reviewed changes
Copy link
Copy Markdown
Member

@maxibor maxibor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's it for me, but there are so many changes, that I most certainly missed some

Comment thread main.nf Outdated
Comment thread main.nf
Comment thread main.nf
Comment thread main.nf Outdated
Comment thread main.nf Outdated
Comment thread main.nf
Comment thread main.nf
Comment thread main.nf Outdated
Comment thread main.nf Outdated
Comment thread main.nf Outdated
@jfy133 jfy133 requested a review from maxibor May 15, 2020 20:09
@jfy133 jfy133 merged commit 3cf7872 into dev May 18, 2020
@jfy133 jfy133 deleted the tsv-input branch May 18, 2020 10:19
This was referenced May 25, 2020
Closed
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@apeltzer apeltzer Awaiting requested review from apeltzer

@drpatelh drpatelh Awaiting requested review from drpatelh

@maxulysse maxulysse Awaiting requested review from maxulysse

@maxibor maxibor Awaiting requested review from maxibor

Assignees

@apeltzer apeltzer

@jfy133 jfy133

Labels

enhancement New feature or request major

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Metadata Sheet Discussion & Multilane merging

4 participants

@jfy133 @maxibor @apeltzer @ZandraFagernas
⚡