SciLifeLab · maxulysse · Dec 17, 2018 · Dec 11, 2018 · Dec 12, 2018 · Dec 12, 2018
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -14,14 +14,15 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 -   [#679](https://github.com/SciLifeLab/Sarek/pull/679) - Add container for `CreateIntervalBeds`
 -   [#692](https://github.com/SciLifeLab/Sarek/pull/692), [#697](https://github.com/SciLifeLab/Sarek/pull/697) - Add AWS iGenomes possibilities (within `conf/igenomes.conf`)
 -   [#694](https://github.com/SciLifeLab/Sarek/pull/694) - Add monochrome and grey logos for light or dark background
+-   [#698](https://github.com/SciLifeLab/Sarek/pull/698) - Add btb profile for munin server
 -   [#702](https://github.com/SciLifeLab/Sarek/pull/702) - Add font-ttf-dejavu-sans-mono `2.37` and fontconfig `2.12.6` to container
 
 ### `Changed`
 
 -   [#678](https://github.com/SciLifeLab/Sarek/pull/678) - Changing VEP to v92 and adjusting CPUs for VEP
 -   [#663](https://github.com/SciLifeLab/Sarek/pull/663) - Update `do_release.sh` script
 -   [#671](https://github.com/SciLifeLab/Sarek/pull/671) - publishDir modes are now params
--   [#677](https://github.com/SciLifeLab/Sarek/pull/677) - Update docs
+-   [#677](https://github.com/SciLifeLab/Sarek/pull/677), [#698](https://github.com/SciLifeLab/Sarek/pull/698) - Update docs
 -   [#679](https://github.com/SciLifeLab/Sarek/pull/679) - Update old awsbatch configuration
 -   [#682](https://github.com/SciLifeLab/Sarek/pull/682) - Specifications for memory and cpus for awsbatch
 -   [#693](https://github.com/SciLifeLab/Sarek/pull/693) - Qualimap bamQC is now ran after mapping and after recalibration for better QC

diff --git a/conf/munin.config b/conf/munin.config
@@ -0,0 +1,140 @@
+/*
+ * -------------------------------------------------
+ * Nextflow config file for Sarek
+ * -------------------------------------------------
+ * Configuration for running on munin
+ * -------------------------------------------------
+ */
+
+env {
+  NXF_WORK="/scratch"
+}
+
+params {
+  genome_base = params.genome == 'GRCh37' ? '/btb/references/Homo_sapiens/GATK/GRCh37/' : params.genome == 'GRCh38' ? '/btb/references/Homo_sapiens/GATK/GRCh38/' : 'References/smallGRCh37'
+  singleCPUMem  = 15.GB
+  totalMemory = 754.GB
+}
+
+executor.$local.cpus = 48
+
+process {
+  // Default process resources
+
+  // A process may use one core,
+  cpus = 1
+
+  // 15 GB of memory,
+  memory = {params.singleCPUMem}
+
+  // and 48 of them are allowed to be launched simultaneously.
+  maxForks = 48
+
+  errorStrategy = {task.exitStatus == 143 ? 'retry' : 'terminate'}
+  maxErrors = '-1'
+  maxRetries = 3
+
+  // These processes are defined in buildReferences.nf
+
+  withName:BuildBWAindexes {
+    memory = {params.totalMemory} // TODO This is likely too high
+  }
+  withName:BuildReferenceIndex {
+    memory = {params.totalMemory} // TODO This is likely too high
+  }
+  withName:BuildSAMToolsIndex {
+    memory = {params.totalMemory} // TODO This is likely too high
+  }
+  withName:BuildVCFIndex {
+    memory = {params.totalMemory} // TODO This is likely too high
+  }
+
+  // These processes are defined in main.nf
+
+  withName:ConcatVCF {
+    cpus = 8
+  }
+  withName:CreateRecalibrationTable {
+    cpus = 16
+    memory = {params.totalMemory}
+  }
+  withName:MapReads {
+    cpus = 48
+    memory = {params.totalMemory}
+  }
+  withName:MarkDuplicates {
+		// Actually the -Xmx value should be kept lower
+    cpus = 16
+    memory = {2 * params.singleCPUMem}
+  }
+  withName:MergeBams {
+    cpus = 8
+    memory = {params.totalMemory}
+  }
+  withName:RecalibrateBam {
+    memory = {params.singleCPUMem * task.attempt}
+  }
+  withName:RunAlleleCount {
+    memory = {params.singleCPUMem * 2 * task.attempt}
+  }
+  withName:RunAscat {
+    memory = {params.singleCPUMem * 2 * task.attempt}
+  }
+  withName:RunBamQCmapped {
+    cpus = 16
+    memory = {params.totalMemory}
+  }
+  withName:RunBamQCrecalibrated {
+    cpus = 16
+    memory = {params.totalMemory}
+  }
+  withName:RunBcftoolsStats {
+  }
+  withName:RunConvertAlleleCounts {
+    memory = {params.singleCPUMem * 2 * task.attempt}
+  }
+  withName:RunFastQC {
+    cpus = 2 // FastQC is only capable of running one thread per fastq file.
+    errorStrategy = {task.exitStatus == 143 ? 'retry' : 'ignore'}
+  }
+  withName:RunFreeBayes {
+    memory = {params.singleCPUMem * task.attempt}
+  }
+  withName:RunHaplotypecaller {
+    // Increase memory quadratically
+    memory = {params.singleCPUMem * task.attempt * task.attempt}
+  }
+  withName:RunGenotypeGVCFs {
+  }
+  withName:RunManta {
+    cpus = 48
+    memory = {params.totalMemory}
+  }
+  withName:RunMultiQC {
+  }
+  withName:RunMutect2 {
+    memory = {params.singleCPUMem * task.attempt}
+  }
+  withName:RunSamtoolsStats {
+  }
+  withName:RunSingleManta {
+    cpus = 48
+    memory = {params.totalMemory}
+  }
+  withName:RunSingleStrelka {
+    cpus = 48
+    memory = {params.totalMemory}
+  }
+  withName:RunSnpeff {
+    errorStrategy = {task.exitStatus == 143 ? 'retry' : 'ignore'}
+    memory = {params.singleCPUMem * task.attempt}
+  }
+  withName:RunStrelka {
+    cpus = 48
+    memory = {params.totalMemory}
+  }
+  withName:RunVEP {
+    cpus = 48
+    errorStrategy = {task.exitStatus == 143 ? 'retry' : 'ignore'}
+  }
+}
diff --git a/docs/CONFIG.md b/docs/CONFIG.md
@@ -13,6 +13,22 @@ The standard ones are designed to work on a Swedish UPPMAX cluster, but can be m
 Every configuration file can be modified for your own use.
 If you want you can specify the use of a config file using `-c <config file>`
 
+### [`aws-batch.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/aws-batch.config)
+
+Designed for usage with AWS batch.
+
+### [`base.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/base.config)
+
+Define default parameters, is included into every profiles.
+
+### [`binac.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/binac.config)
+
+Define usage limits and Singularity for BINAC cluster in Tuebingen.
+
+### [`cfc.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/cfc.config)
+
+Designed for usage with Singularity on CFC at QBic.
+
 ### [`containers.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/containers.config)
 
 Define Containers for all process.
@@ -21,15 +37,27 @@ Use in your own profile if needed.
 
 ### [`docker.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/docker.config)
 
-Define Docker Containers for all process.
-Images will be pulled automatically.
-Use in your own profile if needed.
+Specify Docker options.
+To be used with [`containers.config`](#containersconfig)
 
 ### [`genomes.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/genomes.config)
 
 Contain path to all references.
 Modify it if you want to change genome version, or the path to your references files.
 
+### [`igenomes.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/igenomes.config)
+
+Contain path to all AWS iGenomes references.
+Modify it if you want to change genome version, or the path to your references files.
+
+### [`munin.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/munin.config)
+
+Define usage limits and Singularity for munin server at BTB.
+
+### [`resources.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/resources.config)
+
+Define Generalized resource configuration for clusters.
+
 ### [`singularity-path.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/singularity-path.config)
 
 Define path to Singularity Containers for all process.
@@ -39,14 +67,17 @@ You need to set them up before.
 
 ### [`singularity.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/singularity.config)
 
-Define Singularity Containers for all process.
-Images will be pulled automatically.
-Use in your own profile if needed.
+Specify Singularity options.
+To be used with [`containers.config`](#containersconfig)
 
 ### [`travis.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/travis.config)
 
 To be used for Travis (2 cpus) or on small computer for testing purpose
 
+### [`uppmax-localhost.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/uppmax-localhost.config)
+
+Local configuration for a UPPMAX cluster
+To be run on a single node
 ### [`uppmax-slurm.config`](https://github.com/SciLifeLab/Sarek/blob/master/conf/uppmax-slurm.config)
 
 Slurm configuration for a UPPMAX cluster
@@ -59,17 +90,47 @@ The default profile is `standard`, but Sarek has multiple predefined profiles wh
 ```bash
 nextflow run SciLifeLab/Sarek --sample mysample.tsv -profile myprofile
 ```
+awsbatch {
+binac {
+btb {
+cfc {
+docker {
+singularity {
+singularityPath {
+slurm {
+slurmDownload {
+standard {
+
+### `awsbatch`
 
+This is the profile for use with AWS Batch.
+
+### `binac`
+
+This is the profile for use on the german BinAC cluster.
+
+### `btb`
+
+This is the profile for use on the BTB server munin.
+
+### `cfc`
+
+This is the profile for use on the CFC cluster in Tuebingen.
 
 ### `docker`
 
 This is the profile for docker testing on a small machine, or on Travis CI.
 Docker images will be pulled automatically.
 
-### `standard`
+### `singularity`
 
-This is the default profile for use on a localhost on a UPPMAX cluster with Singularity.
-Singularity images need to be set up.
+This is the profile for Singularity testing on a small machine, or on Travis CI.
+Singularity images will be pulled automatically.
+
+### `singularityPath`
+
+This is the profile for Singularity testing on a small machine.
+Singularity images needs to be set up.
 
 ### `slurm`
 
@@ -83,10 +144,10 @@ This is another profile for use on a UPPMAX cluster using the job scheduler slur
 Will run the workflow on `/scratch`.
 Singularity images will be pulled automatically.
 
-### `singularity`
+### `standard`
 
-This is the profile for Singularity testing on a small machine, or on Travis CI.
-Singularity images will be pulled automatically.
+This is the default profile for use on a localhost on a UPPMAX cluster with Singularity.
+Singularity images need to be set up.
 
 ## Customisation
 The recommended way to use custom settings is to supply Sarek with an additional configuration file. You can use the files in the [`conf/`](https://github.com/SciLifeLab/Sarek/tree/master/conf) directory as an inspiration to make this new `.config` file and specify it using the `-c` flag:

diff --git a/docs/REFERENCES.md b/docs/REFERENCES.md
@@ -101,7 +101,7 @@ Use `--genome smallGRCh37` to map against a small reference genome based on GRCh
 
 ## AWS iGenomes
 Sarek is using [AWS iGenomes](https://ewels.github.io/AWS-iGenomes/), which facilitate storing and sharing references.
-Both `GRCh37` and `GRCh38` are available with `--genome GRCh37` or `--genome GRCh38` respectively with any profile using the `conf/igenomes.config` file (eg.: `awsbatch`), or you can specify it with `-c conf/igenomes.config`, it contains all data previously detailed.
+Both `GRCh37` and `GRCh38` are available with `--genome GRCh37` or `--genome GRCh38` respectively with any profile using the `conf/igenomes.config` file (eg.: `awsbatch`, or `btb`), or you can specify it with `-c conf/igenomes.config`, it contains all data previously detailed.
 
 ## buildReferences.nf