Cpus=1 necessary for workflow_summary_mqc?

[alneberg]

I got an error running the dev version of the pipeline. It failed with the error:

ERROR ~ Error executing process > 'workflow_summary_mqc'

Caused by:
  Process requirement exceed available CPUs -- req: 2; avail: 1

when running:

$ nextflow run nf-core/rnaseq --project snic2017-7-196 --singleEnd --reads '../test_data/SRX22032*_first10000.fastq.gz' --genome GRCm38 -profile uppmax_devel -name test_run_5 -r dev -resume

I found something that looks like a configuration (cpus=1) of the local executor this in the log file:

Dec-07 13:46:28.169 [main] DEBUG nextflow.processor.ProcessFactory - << taskConfig executor: local
Dec-07 13:46:28.169 [main] DEBUG nextflow.processor.ProcessFactory - >> processorType: 'local'
Dec-07 13:46:28.171 [main] DEBUG nextflow.executor.Executor - Initializing executor: local
Dec-07 13:46:28.171 [main] INFO  nextflow.executor.Executor - [warm up] executor > local
Dec-07 13:46:28.174 [main] DEBUG n.processor.LocalPollingMonitor - Creating local task monitor for executor 'local' > cpus=1; memory=251.4 GB; capacity=1; pollInterval=100ms; dumpInterval=5m

So I was able to resolve the issue by supplying the following extra config file (with -c):

process {
  withName:workflow_summary_mqc {
    cpus = 1
  }
}

But I'm not sure if this is a necessary change for base.config or if it is possible that the local executor is configured wrongly somehow on our cluster?

Here is the complete Nextflow output:

N E X T F L O W  ~  version 18.10.1                                                                                                                                                                                                  [240/890]
Launching `nf-core/rnaseq` [test_run_5] - revision: 565a8a5e21 [dev]
=======================================================
                                          ,--./,-.
          ___     __   __   __   ___     /,-._.--~'
    |\ | |__  __ /  ` /  \ |__) |__         }  {
    | \| |       \__, \__/ |  \ |___     \`-._,-`-,
                                          `._,._,'

 nf-core/rnaseq : RNA-Seq Best Practice v1.1
=======================================================
Run Name       : test_run_5
Reads          : ../test_data/SRX22032*_first10000.fastq.gz
Data Type      : Single-End
Genome         : GRCm38
Strandedness   : None
Trim R1        : 0
Trim R2        : 0
Trim 3' R1     : 0
Trim 3' R2     : 0
Aligner        : STAR
STAR Index     : /sw/data/uppnex/igenomes//Mus_musculus/Ensembl/GRCm38/Sequence/STARIndex/
GTF Annotation : /sw/data/uppnex/igenomes//Mus_musculus/Ensembl/GRCm38/Annotation/Genes/genes.gtf
BED Annotation : /sw/data/uppnex/igenomes//Mus_musculus/Ensembl/GRCm38/Annotation/Genes/genes.bed
Save Reference : Yes
Save Trimmed   : No
Save Intermeds : No
Max Memory     : 120 GB
Max CPUs       : 16
Max Time       : 1h
Output dir     : ./results
Working dir    : /crex/proj/sllstore2017079/private/johannes/rnaseq_test_run/work
Container      : nfcore/rnaseq:latest
Pipeline Release: dev
Current home   : /home/alneberg
Current user   : alneberg
Current path   : /proj/sllstore2017079/private/johannes/rnaseq_test_run
Script dir     : /home/alneberg/.nextflow/assets/nf-core/rnaseq
Config Profile : uppmax_devel
UPPMAX Project : snic2017-7-196
=========================================
[warm up] executor > slurm
WARN: Singularity cache directory has not been defined -- Remote image will be stored in the path: /crex/proj/sllstore2017079/private/johannes/rnaseq_test_run/work/singularity
Pulling Singularity image docker://nfcore/rnaseq:latest [cache /crex/proj/sllstore2017079/private/johannes/rnaseq_test_run/work/singularity/nfcore-rnaseq-latest.img]
[warm up] executor > local
ERROR ~ Error executing process > 'workflow_summary_mqc'

Caused by:
  Process requirement exceed available CPUs -- req: 2; avail: 1

Source block:
      def yaml_file = task.workDir.resolve('workflow_summary_mqc.yaml')
      yaml_file.text  = """
      id: 'nfcore-rnaseq-summary'
      description: " - this information is collected when the pipeline is started."
      section_name: 'nfcore/rnaseq Workflow Summary'
      section_href: 'https://github.com/nf-core/rnaseq'
      plot_type: 'html'
      data: |
          <dl class=\"dl-horizontal\">
  ${summary.collect { k,v -> "            <dt>$k</dt><dd><samp>${v ?: '<span style=\"color:#999999;\">N/A</a>'}</samp></dd>" }.join("\n")}
          </dl>
      """.stripIndent()

Work dir:
  /crex/proj/sllstore2017079/private/johannes/rnaseq_test_run/work/1f/92998cf77032f99642b424e7f32d2f

Tip: when you have fixed the problem you can continue the execution appending to the nextflow command line the option `-resume`

 -- Check '.nextflow.log' file for details
[nfcore/rnaseq] Pipeline Complete
[1b/48222f] Submitted process > fastqc (SRX2203288)

[ewels]

Probably not worth debugging - if cpus 1 works, then let's just use that.

rnaseq/conf/base.config

Line 93 in 1cd5ab7

memory = { check_max( 2.GB, 'memory' ) }

[apeltzer]

Added a PR to fix this: #138

[apeltzer]

Fixed in #138, will be in the next release