add support for running shell commands asynchronously with run_shell_cmd

[boegel]

TODO:

• add support in run_shell_cmd for using asynchronous=True
• use run_shell_cmd with asynchronous=True in EasyBlock.skip_extensions_parallel
• use run_shell_cmd with asynchronous=True in Extension.async_cmd_start
• test with R easyconfig and --parallel-extensions-install (and --skip)

[boegel]

Quick test:

• eb R-4.2.2-foss-2022b.eb --skip --force (serial) took ~28m30s
• eb R-4.2.2-foss-2022b.eb --skip --parallel-ext --experimental -f with 24 cores (RHEL8, AMD Rome) took ~16m55s (with EasyBuild 4.9.x it takes ~18m20s)

And that's only the skipping part being done in parallel...

[boegel]

I've tested this with R-4.2.1-foss-2022a.eb, and saw a significant speedup when using --parallel-extensions-install --experimental using 24 cores (RHEL8, AMD Rome):

• with sequential installation of extensions (default):
  • 6h25min total time
  • 5h50min for extensions step
• with --parallel-extensions-install --experimental:
  • 1h54min total time => 3.37x speedup
  • 1h21min for extensions step => 4.32x speedup for this specific step

The scaling is not spectacular, but I think that's mainly determine by the order in which extensions are being considered, not so much the implementation of running the extension installations in parallel in the background.

So, I think this is ready for review/merge.

Do note that these changes require a corresponding change in easyblocks, see:

• update run_async methods in RPackage, Rmpi, and Rserve easyblocks to use submit run_shell_cmd call into thread pool easybuild-easyblocks#3163

I prefer keeping the support for --parallel-extensions-install experimental for now, because I would like to revisit install_extensions_parallel later and improve things a bit.

There's a small additional change needed to make parallel installation of extensions work for easyconfigs like R-bundle-Bioconductor or R-bundle-CRAN, but I'll make that change in a separate PR, I don't want to bury that in this one...

[lexming]

LGTM