parameter to control whether sequential or threaded versions of linalg libraries will be used?

[migueldiascosta]

(prompted by easybuilders/easybuild-easyblocks#1294)

should there be a default parameter to control linking against sequential or threaded blas/lapack libraries?

if that parameter set, e.g., $LIB{BLAS/LAPACK/SCALAPACK} to the value of $LIB{BLAS/LAPACK/SCALAPACK}_MT, there would be no need to change the easyblocks

admitedly, it would change the current behaviour, but only in the cases where the new parameter is set

[boegel]

I like this idea, and as long as we don't change default behaviour, that's fine imho, since that retain backward compatibility.

This should be a new toolchain option, so this can be set on demand in specific easyconfig files?

[akesandgren]

Basically if toolchainopt has openmp the _MT libs should be used.
Unfortunately that doesn't always apply. There are some openmp codes that must use the non-threaded libs since they do the openmp part on a higher level (this for performance reasons mostly)

But i do agree that it would be useful to have such an option. There are a lot of codes that would benefit from that performance wise.

[migueldiascosta]

@akesandgren at first I also thought that the openmp toolchainopt could be used. In most cases, either the application itself is not openmp parallelized so it's irrelevant, or you actually want to use both threaded application and libraries.

The cases I see where one would like to enable threaded libraries without enabling openmp in an openmp parallelized application would be if that parallelization is buggy, actually slower than serial, or if it would imply nested openmp regions, as you mention.

As for benefits, as far as I'm concerned we mostly run MPI applications with one process in every available core (and no hyperthreading) so we use sequential libraries, but there are a few cases, e.g. for reducing memory usage and/or communication, where reducing the number of processes and using threads on the otherwise idle cores may be useful. This may be more relevant in KNL nodes, because of clustering modes, hbm, hyperthreading?

Saw in the meeting notes that someone mentioned this would also apply to FFT, which would imply yet another option. Could the new toolchainopt be a list of libraries to override, e.g. use_mt='blas,lapack,fft'?

[akesandgren]

Yes, that's even better, defining a list is good. The question is how to define the libs to make it easy to use and easy to code in the framework.

[boegel]

I finally got around to finish the PR I had in the works for this, see #2356.

For now, it only adds the (boolean) mt_blas_lapack toolchain option, which is used as trigger to use for value of all $*MT environment variables for their non-MT equivalents (so $LIBBLAS is defined like $LIBBLAS_MT, etc.).

W.r.t. the list idea, I guess that makes sense, but I'd name it use_mt_libs?
Allowed values would then be blas, lapack, scalapack, fft?

Currently we don't have $LIBFFT_MT yet though, only $LIBFFT.
Did you have an issue (or PR) for this @akesandgren?

I suggest we follow up in there, "in context"?