Merge in Template V1.6.

.github/CONTRIBUTING.md

@@ -6,7 +6,9 @@ We try to manage the required tasks for nf-core/eager using GitHub issues, you p
 
 However, don't be put off by this template - other more general issues and suggestions are welcome! Contributions to the code are even more welcome ;)
 
-> If you need help using or modifying nf-core/eager then the best place to go is the Gitter chatroom where you can ask us questions directly: https://gitter.im/nf-core/Lobby
+> If you need help using or modifying nf-core/eager then the best place to ask is on the pipeline channel on [Slack](https://nf-core-invite.herokuapp.com/).
+
+
 
 ## Contribution workflow
 If you'd like to write some code for nf-core/eager, the standard workflow
@@ -30,7 +32,7 @@ Typically, pull-requests are only fully reviewed when these tests are passing, t
 There are typically two types of tests that run:
 
 ### Lint Tests
-The nf-core has a [set of guidelines](http://nf-co.re/developer_docs) which all pipelines must adhere to.
+The nf-core has a [set of guidelines](http://nf-co.re/guidelines) which all pipelines must adhere to.
 To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.
 
 If any failures or warnings are encountered, please follow the listed URL for more documentation.
@@ -42,4 +44,4 @@ If there are any failures then the automated tests fail.
 These tests are run both with the latest available version of Nextflow and also the minimum required version that is stated in the pipeline code.
 
 ## Getting help
-For further information/help, please consult the [nf-core/eager documentation](https://github.com/nf-core/eager#documentation) and don't hesitate to get in touch on [Gitter](https://gitter.im/nf-core/Lobby)
+For further information/help, please consult the [nf-core/eager documentation](https://github.com/nf-core/eager#documentation) and don't hesitate to get in touch on the pipeline channel on [Slack](https://nf-core-invite.herokuapp.com/).

.github/markdownlint.yml

@@ -0,0 +1,9 @@
+# Markdownlint configuration file
+default: true,
+line-length: false
+no-multiple-blanks: 0
+blanks-around-headers: false
+blanks-around-lists: false
+header-increment: false
+no-duplicate-header:
+    siblings_only: true

.gitignore

@@ -4,3 +4,4 @@ data/
 results/
 .DS_Store
 tests/test_data
+*.pyc

.travis.yml

@@ -11,19 +11,23 @@ before_install:
   # PRs to master are only ok if coming from dev branch
   - '[ $TRAVIS_PULL_REQUEST = "false" ] || [ $TRAVIS_BRANCH != "master" ] || ([ $TRAVIS_PULL_REQUEST_SLUG = $TRAVIS_REPO_SLUG ] && [ $TRAVIS_PULL_REQUEST_BRANCH = "dev" ])'
   # Pull the docker image first so the test doesn't wait for this
-  - docker pull nfcore/eager
+  - docker pull nfcore/eager:dev
   # Fake the tag locally so that the pipeline runs properly
-  - docker tag nfcore/eager nfcore/eager:latest
+  # Looks weird when this is :dev to :dev, but makes sense when testing code for a release (:dev to :1.0.1)
+  - docker tag nfcore/eager:dev nfcore/eager:dev
 
 install:
   # Install Nextflow
   - mkdir /tmp/nextflow && cd /tmp/nextflow
   - wget -qO- get.nextflow.io | bash
   - sudo ln -s /tmp/nextflow/nextflow /usr/local/bin/nextflow
   # Install nf-core/tools
+  - pip install --upgrade pip
   - pip install nf-core
   # Reset
   - mkdir ${TRAVIS_BUILD_DIR}/tests && cd ${TRAVIS_BUILD_DIR}/tests
+  # Install markdownlint-cli
+  - sudo apt-get install npm && npm install -g markdownlint-cli
 
 env:
   - NXF_VER='0.32.0' # Specify a minimum NF version that should be tested and work
@@ -32,5 +36,7 @@ env:
 script:
   # Lint the pipeline code
   - nf-core lint ${TRAVIS_BUILD_DIR}
+  # Lint the documentation
+  - markdownlint ${TRAVIS_BUILD_DIR} -c ${TRAVIS_BUILD_DIR}/.github/markdownlint.yml
   # Run the pipeline with the test profile
   - nextflow run ${TRAVIS_BUILD_DIR} -profile test,docker

CHANGELOG.md

@@ -1,4 +1,4 @@
 # nf-core/eager: Changelog
 
-## v1.0dev - <date>
+## v2.0.6 - [date]
 Initial release of nf-core/eager, created with the [nf-core](http://nf-co.re/) template.

CODE_OF_CONDUCT.md

@@ -34,7 +34,7 @@ This Code of Conduct applies both within project spaces and in public spaces whe
 
 ## Enforcement
 
-Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on the [Gitter channel](https://gitter.im/nf-core/Lobby). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
+Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by contacting the project team on [Slack](https://nf-core-invite.herokuapp.com/). The project team will review and investigate all complaints, and will respond in a way that it deems appropriate to the circumstances. The project team is obligated to maintain confidentiality with regard to the reporter of an incident. Further details of specific enforcement policies may be posted separately.
 
 Project maintainers who do not follow or enforce the Code of Conduct in good faith may face temporary or permanent repercussions as determined by other members of the project's leadership.
 

Dockerfile

@@ -1,6 +1,7 @@
 FROM nfcore/base
-LABEL description="Docker image containing all requirements for nf-core/eager pipeline"
+LABEL authors="Alexander Peltzer, Stephen Clayton, James A Fellows-Yates" \
+      description="Docker image containing all requirements for nf-core/eager pipeline"
 
 COPY environment.yml /
 RUN conda env create -f /environment.yml && conda clean -a
-ENV PATH /opt/conda/envs/nf-core-eager-1.0dev/bin:$PATH
+ENV PATH /opt/conda/envs/nf-core-eager-2.0.6/bin:$PATH

LICENSE

@@ -1,5 +1,7 @@
 MIT License
 
+Copyright (c) Alexander Peltzer, Stephen Clayton, James A Fellows-Yates
+
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal
 in the Software without restriction, including without limitation the rights

README.md

@@ -1,25 +1,30 @@
 # nf-core/eager
-**A fully reproducible aDNA analysis pipeline.**
 
-[![Build Status](https://travis-ci.org/nf-core/eager.svg?branch=master)](https://travis-ci.org/nf-core/eager)
+**A fully reproducible and modern ancient DNA pipeline in Nextflow and with cloud support.**.
+
+[![Build Status](https://travis-ci.com/nf-core/eager.svg?branch=master)](https://travis-ci.com/nf-core/eager)
 [![Nextflow](https://img.shields.io/badge/nextflow-%E2%89%A50.32.0-brightgreen.svg)](https://www.nextflow.io/)
 
 [![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg)](http://bioconda.github.io/)
 [![Docker](https://img.shields.io/docker/automated/nfcore/eager.svg)](https://hub.docker.com/r/nfcore/eager)
-![Singularity Container available](
-https://img.shields.io/badge/singularity-available-7E4C74.svg)
 
-### Introduction
-The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker / singularity containers making installation trivial and results highly reproducible.
+## Introduction
+The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It comes with docker containers making installation trivial and results highly reproducible.
 
 
-### Documentation
+## Documentation
 The nf-core/eager pipeline comes with documentation about the pipeline, found in the `docs/` directory:
 
-1. [Installation](docs/installation.md)
+1. [Installation](https://nf-co.re/usage/installation)
 2. Pipeline configuration
-    * [Local installation](docs/configuration/local.md)
-    * [Adding your own system](docs/configuration/adding_your_own.md)
+    * [Local installation](https://nf-co.re/usage/local_installation)
+    * [Adding your own system config](https://nf-co.re/usage/adding_own_config)
+    * [Reference genomes](https://nf-co.re/usage/reference_genomes)
 3. [Running the pipeline](docs/usage.md)
 4. [Output and how to interpret the results](docs/output.md)
-5. [Troubleshooting](docs/troubleshooting.md)
+5. [Troubleshooting](https://nf-co.re/usage/troubleshooting)
+
+<!-- TODO nf-core: Add a brief overview of what the pipeline does and how it works -->
+
+## Credits
+nf-core/eager was originally written by Alexander Peltzer, Stephen Clayton, James A Fellows-Yates.

assets/email_template.html

@@ -5,7 +5,7 @@
   <meta http-equiv="X-UA-Compatible" content="IE=edge">
   <meta name="viewport" content="width=device-width, initial-scale=1">
 
-  <meta name="description" content="nf-core/eager: A fully reproducible aDNA analysis pipeline.">
+  <meta name="description" content="nf-core/eager: A fully reproducible and modern ancient DNA pipeline in Nextflow and with cloud support.">
   <title>nf-core/eager Pipeline Report</title>
 </head>
 <body>

assets/email_template.txt

@@ -17,23 +17,6 @@ ${errorReport}
 } %>
 
 
-<% if (!success){
-    out << """####################################################
-## nf-core/eager execution completed unsuccessfully! ##
-####################################################
-The exit status of the task that caused the workflow execution to fail was: $exitStatus.
-The full error message was:
-
-${errorReport}
-"""
-} else {
-    out << "## nf-core/eager execution completed successfully! ##"
-}
-%>
-
-
-
-
 The workflow was completed at $dateComplete (duration: $duration)
 
 The command used to launch the workflow was as follows:

conf/multiqc_config.yaml → assets/multiqc_config.yaml

@@ -5,3 +5,5 @@ report_comment: >
 report_section_order:
     nf-core/eager-software-versions:
         order: -1000
+
+export_plots: true

assets/sendmail_template.txt

@@ -1,11 +1,36 @@
 To: $email
 Subject: $subject
 Mime-Version: 1.0
-Content-Type: multipart/related;boundary="nfmimeboundary"
+Content-Type: multipart/related;boundary="nfcoremimeboundary"
 
---nfmimeboundary
+--nfcoremimeboundary
 Content-Type: text/html; charset=utf-8
 
 $email_html
 
---nfmimeboundary--
+<%
+if (mqcFile){
+def mqcFileObj = new File("$mqcFile")
+if (mqcFileObj.length() < mqcMaxSize){
+out << """
+--nfcoremimeboundary
+Content-Type: text/html; name=\"multiqc_report\"
+Content-Transfer-Encoding: base64
+Content-ID: <mqcreport>
+Content-Disposition: attachment; filename=\"${mqcFileObj.getName()}\"
+
+${mqcFileObj.
+  bytes.
+  encodeBase64().
+  toString().
+  tokenize( '\n' )*.
+  toList()*.
+  collate( 76 )*.
+  collect { it.join() }.
+  flatten().
+  join( '\n' )}
+"""
+}}
+%>
+
+--nfcoremimeboundary--

bin/scrape_software_versions.py

@@ -3,6 +3,7 @@
 from collections import OrderedDict
 import re
 
+# TODO nf-core: Add additional regexes for new tools in process get_software_versions
 regexes = {
     'nf-core/eager': ['v_pipeline.txt', r"(\S+)"],
     'Nextflow': ['v_nextflow.txt', r"(\S+)"],
@@ -23,9 +24,14 @@
         if match:
             results[k] = "v{}".format(match.group(1))
 
+# Remove software set to false in results
+for k in results:
+    if not results[k]:
+        del(results[k])
+
 # Dump to YAML
 print ('''
-id: 'nf-core/eager-software-versions'
+id: 'software_versions'
 section_name: 'nf-core/eager Software Versions'
 section_href: 'https://github.com/nf-core/eager'
 plot_type: 'html'
@@ -34,5 +40,10 @@
     <dl class="dl-horizontal">
 ''')
 for k,v in results.items():
-    print("        <dt>{}</dt><dd>{}</dd>".format(k,v))
+    print("        <dt>{}</dt><dd><samp>{}</samp></dd>".format(k,v))
 print ("    </dl>")
+
+# Write out regexes as csv file:
+with open('software_versions.csv', 'w') as f:
+    for k,v in results.items():
+        f.write("{}\t{}\n".format(k,v))

conf/awsbatch.config

@@ -1,10 +1,15 @@
 /*
  * -------------------------------------------------
- *  Nextflow config file for AWS Batch
+ *  Nextflow config file for running on AWS batch
  * -------------------------------------------------
- * Imported under the 'awsbatch' Nextflow profile in nextflow.config
- * Uses docker for software depedencies automagically, so not specified here.
+ * Base config needed for running with -profile awsbatch
  */
+params {
+  config_profile_name = 'AWSBATCH'
+  config_profile_description = 'AWSBATCH Cloud Profile'
+  config_profile_contact = 'Alexander Peltzer (@apeltzer)'
+  config_profile_url = 'https://aws.amazon.com/de/batch/'
+}
 
 aws.region = params.awsregion
 process.executor = 'awsbatch'

conf/base.config

@@ -11,23 +11,18 @@
 
 process {
 
-  container = params.container
-
+  // TODO nf-core: Check the defaults for all processes
   cpus = { check_max( 1 * task.attempt, 'cpus' ) }
   memory = { check_max( 8.GB * task.attempt, 'memory' ) }
   time = { check_max( 2.h * task.attempt, 'time' ) }
 
-  errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'finish' }
+  errorStrategy = { task.exitStatus in [143,137,104,134,139] ? 'retry' : 'finish' }
   maxRetries = 1
   maxErrors = '-1'
 
   // Process-specific resource requirements
-  withName: fastqc {
-    errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'ignore' }
-  }
-  withName: multiqc {
-    errorStrategy = { task.exitStatus in [143,137] ? 'retry' : 'ignore' }
-  }
+  // TODO nf-core: Customise requirements for specific processes.
+  // See https://www.nextflow.io/docs/latest/config.html#config-process-selectors
 }
 
 params {

conf/igenomes.config

@@ -9,6 +9,7 @@
 
 params {
   // illumina iGenomes reference file paths
+  // TODO nf-core: Add new reference types and strip out those that are not needed
   genomes {
     'GRCh37' {
       bed12   = "${params.igenomes_base}/Homo_sapiens/Ensembl/GRCh37/Annotation/Genes/genes.bed"

conf/test.config

@@ -4,14 +4,19 @@
  * -------------------------------------------------
  * Defines bundled input files and everything required
  * to run a fast and simple test. Use as follows:
- *   nextflow run nf-core/methylseq -profile test
+ *   nextflow run nf-core/eager -profile test
  */
 
 params {
+  config_profile_name = 'Test profile'
+  config_profile_description = 'Minimal test dataset to check pipeline function'
+  // Limit resources so that this can run on Travis
   max_cpus = 2
   max_memory = 6.GB
   max_time = 48.h
   // Input data
+  // TODO nf-core: Specify the paths to your test data on nf-core/test-datasets
+  // TODO nf-core: Give any required params for the test so that command line flags are not needed
   singleEnd = false
   readPaths = [
     ['Testdata', ['https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R1.tiny.fastq.gz', 'https://github.com/nf-core/test-datasets/raw/exoseq/testdata/Testdata_R2.tiny.fastq.gz']],

docs/README.md

@@ -2,10 +2,11 @@
 
 The nf-core/eager documentation is split into the following files:
 
-1. [Installation](installation.md)
-2. [Running the pipeline](usage.md)
-3. Pipeline configuration
-    * [Adding your own system](configuration/adding_your_own.md)
-    * [Reference genomes](configuration/reference_genomes.md)
+1. [Installation](https://nf-co.re/usage/installation)
+2. Pipeline configuration
+    * [Local installation](https://nf-co.re/usage/local_installation)
+    * [Adding your own system config](https://nf-co.re/usage/adding_own_config)
+    * [Reference genomes](https://nf-co.re/usage/reference_genomes)
+3. [Running the pipeline](usage.md)
 4. [Output and how to interpret the results](output.md)
-5. [Troubleshooting](troubleshooting.md)
+5. [Troubleshooting](https://nf-co.re/usage/troubleshooting)