Conference call notes 20170201

(back to Conference calls)

Notes on the 69th EasyBuild conference call, Wednesday February 1st 2017 (5pm - 6pm CET)

Attendees

Alphabetical list of attendees (7):

• Damian Alvarez (JSC, Germany)
• Markus Geimer (JSC, Germany)
• Kenneth Hoste (HPC-UGent, Belgium)
• Alan O'Cais (JSC, Germany)
• Åke Sandgren (Umeå University, Sweden)
• Shahzeb Siddiqui (Pfizer)
• Bart Oldeman (McGill University, Canada)

Agenda

• foss/2017a & intel/2017a toolchain definitions set in stone
• EasyBuild v3.1.0 will be released later this week
• performance issues of out-of-the-box MPI installations (& beyond)
• Q&A

Notes

2017a toolchains

• foss/2017a: https://github.com/hpcugent/easybuild-easyconfigs/pull/3968
  • updated to use OpenMPI 2.0.2 & FFTW 3.3.6
  • Markus: two versions of M4/zlib uses in the full dep graph?
    • ...
  • Damian: pkg-config, Qt4, GLib (older version) fails to compile out-of-the-box with GCC 6.3
    • mostly due to changes of C++ standard (GCC 6.3 uses C++14 by default)
    • can be fixed using something like toolchainopts = {'cstd': 'gnu++98'} => g++ -std=gnu++98
    • pkg-config has a dedicated configure option --disable-compile-warnings
• intel/2017a: https://github.com/hpcugent/easybuild-easyconfigs/pull/3969
  • all done, on top of GCC 6.3.0 (same as in foss/2017a)
  • confirmed to work for HPL 2.2, CP2K 3.0, Python 2.7.12, Python 3.5.2, Perl 5.24.0, R/3.3.1

EasyBuild v3.1.0

• framework: https://github.com/hpcugent/easybuild-framework/milestone/40
  • still considering Bart's PR (https://github.com/hpcugent/easybuild-framework/pull/2091)
• easyblocks: https://github.com/hpcugent/easybuild-easyblocks/milestone/31
  • all done
• easyconfigs: https://github.com/hpcugent/easybuild-easyconfigs/milestone/34
  • awaiting final test for 2017a toolchains
  • maybe also applying of trivial patch for XZ to avoid breaking rpm command on CentOS7.3
    • cfr. https://github.com/hpcugent/easybuild-easyconfigs/issues/4036

Performance issues with out-of-the-box MPI installations

• MPI installs done by non-admins using EB (cfr. post on mailing list)
  • without realising it requires further tuning/configuration
  • two aspects:
    • configuring via environment variables, quite site-specific
    • making sure right system libraries are used and pick up by the MPI installation
  • enforce specifying site-specific tweaking for certain stuff like OpenMPI?
    • and print a big ugly warning when nothing is specified at all...
      • could be part of sanity check?
    • this would require a good way to specify site-specific tweak in a separate file next to the easyconfig
    • e.g. for MPICH a detailed 'report' could be print at the sanity check

Other

• Damian: extension support for Octave?

  • see OCaml package support: https://github.com/hpcugent/easybuild-easyblocks/blob/master/easybuild/easyblocks/generic/ocamlpackage.py

• Shahzeb: live streaming/recording of EasyBuild User Meeting?

  • Alan: plan is to do both, more information coming
  • cfr. upcoming post to EasyBuild mailing list early next week

• Shahzeb: combination of EasyBuild and Singularity?

  • see for example https://github.com/pescobar/singularity-easybuild w.r.t. running EasyBuild in a Singularity container
  • other possible use cases:
    • creating Singularity containers with EasyBuild
    • running EasyBuild in isolation from the OS through Singularity
    • testing EasyBuild on different OSs (regardless of host OS)

• installation of DAAL, etc. fails because of missing custom sanity checks

  • https://github.com/hpcugent/easybuild-easyconfigs/issues/4074
  • seems to be just missing a custom sanity_check_paths: http://easybuild.readthedocs.io/en/latest/Writing_easyconfig_files.html#sanity-check
  • some other Intel tool updates may require updated easyblock
    • e.g. Inspector 2017: https://github.com/hpcugent/easybuild-easyblocks/pull/1047

• installing Intel PSXE as a whole vs separate components?

  • installation itself is fairly trivial, using it as a toolchain is less trivial
  • cfr. https://github.com/hpcugent/easybuild-framework/pull/2096
  • intel-psxe could be defined as a separate toolchain
  • still leaves the question on which road to take for intel/2017b in the not-so-distant future
  • preferred way will differ per site
  • e.g., in HMNS setting combining icc & ifort in a single location is preferable

• Shahzeb: support for Intel Python?

  • not supported yet, needs a dedicated easyblock
    • cfr. https://github.com/hpcugent/easybuild-easyconfigs/issues/3922
  • Kenneth: interested to compare Intel Python with Python built with intel toolchain
    • e.g. via https://github.com/IntelPython/ibench
  • Bart: https://dpinte.wordpress.com/2010/01/15/numpy-performance-improvement-with-the-mkl/

• Markus: getting optarch compiler flags always as a string?

  • grabbing it via options_map may result in a string or list value (latter from ibmxl compilers)
  • there's a better way, to always get a (flattened) string...
    • Damian: cfr. self.options.optarch?