Nested Environment Module names are not loaded properly

[apeltzer]

Hi everyone,

I tried executing a Singularity container with a basic script on our PBS cluster. After creating the script, I created a configuration file

process { 
executor = 'pbs' 
module = 'qbic/singularity/2.4' 
container = '/share/test/testcontainer.simg' 
cpus = 8 
memory = 32.GB 
time = 24.h 
queue = 'cfc' 
}

It seems to be problematic, that NF can't find the module we need to load for getting singularity to run on our cluster. I typically load it like this module load qbic/singularity/2.4 or similar.

If I check the '.command.env' file, I find this:

... nxf_module_load qbic singularity 2.4
(which should have / in between of course).

[pditommaso]

The problem is that NF expected the module name as name/version with a single slash.

Unfortunately I don't have an environment to test your condition, could you make this test. Save in a script snippet:

set -x

nxf_module_load(){
  local mod=$1
  local ver=${2:-}
  local new_module="$mod"; [[ $ver ]] && new_module+="/$ver"

  if [[ ! $(module list 2>&1 | grep -o "$new_module") ]]; then
    old_module=$(module list 2>&1 | grep -Eo "$mod\/[^\( \n]+" || true)
    if [[ $ver && $old_module ]]; then
      module switch $old_module $new_module
    else
      module load $new_module
    fi
  fi
}

nxf_module_load qbic/singularity 2.4

then execute it and post the output here?

[apeltzer]

sh test_paolo.sh 
+ nxf_module_load qbic/singularity 2.4 
+ local mod=qbic/singularity
+ local ver=2.4
+ local new_module=qbic/singularity
+ [[ -n 2.4 ]]
+ new_module+=/2.4
++ module list
++ grep -o qbic/singularity/2.4
+ [[ ! -n '' ]]
++ module list
++ grep -Eo 'qbic/singularity\/[^\( \n]+'
++ true
+ old_module=
+ [[ -n 2.4 ]]
+ [[ -n '' ]]
+ module load qbic/singularity/2.4
++ /usr/share/Modules/bin/modulecmd-wrapper bash load qbic/singularity/2.4
+ eval LOADEDMODULES=qbic/singularity/2.4 ';export' 
'LOADEDMODULES;PATH=/lustre_cfc/software/qbic/singularity/2.4/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home-link/iiipe01/bin' ';export' 
'PATH;SINGULARITY_BIN_DIR=/lustre_cfc/software/qbic/singularity/2.4/bin' ';export' 
'SINGULARITY_BIN_DIR;SINGULARITY_VERSION=2.4' ';export' 
'SINGULARITY_VERSION;_LMFILES_=/lustre_cfc/software/qbic/modulefiles/qbic/singularity/2.4' 
';export' '_LMFILES_;'
++ LOADEDMODULES=qbic/singularity/2.4
++ export LOADEDMODULES
++ PATH=/lustre_cfc/software/qbic/singularity/2.4/bin:/usr/lib64/qt-
3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home-link/iiipe01/bin
++ export PATH
++ SINGULARITY_BIN_DIR=/lustre_cfc/software/qbic/singularity/2.4/bin
++ export SINGULARITY_BIN_DIR
++ SINGULARITY_VERSION=2.4
++ export SINGULARITY_VERSION
++ _LMFILES_=/lustre_cfc/software/qbic/modulefiles/qbic/singularity/2.4
++ export _LMFILES_

[pditommaso]

I see module load qbic/singularity/2.4, it seems to work properly in this way, right?

[apeltzer]

Yes! I added a small statement to your script (singularity --version) and it states 2.4-dist which is correct. This should work now 👍

[pditommaso]

OK, providing a patch soon.

[pditommaso]

I've uploaded a new build (4664) that should fix this problem. You may want to give a try prepending the var NXF_VER=0.26.0-SNAPSHOT to usual NF commands, eg

NXF_VER=0.26.0-SNAPSHOT nextflow run .. etc

(the first time it will download the required deps)

[apeltzer]

Can confirm that this resolves the issue !

[apeltzer]

Hmm, still not working. I don't know why it doesn't load the module beforehand. Maybe you see something?

command.env.txt
command.err.txt
command.log.txt
command.run.txt
command.sh.txt

[pditommaso]

Are you loading singularity as a module to run a NF task? that's not possible unfortunately.

[pditommaso]

However it should work by using beforeScript directive.

[apeltzer]

Oh, okay. I thought this was the way to do it. Yes, I thought this would load the module for the execution of the job & then allow me to run anything with Singularity.

I've run into another issue with that: When using the beforeScript directive, the shub command won't work unfortunately. So something like this in the conf:

container = 'shub://qbicsoftware/qbic-singularity-megSAP'

in the config doesn't work either.