add easyblock for RAxML by lexming · Pull Request #2180 · easybuilders/easybuild-easyblocks

lexming · 2020-09-16T17:45:41Z

This easyblock automatically builds those binaries of RAxML that

use optimizations available in the CPU micro-architecture of the host system (respects optarch settings)
have features compatible with the toolchain used in the build

Current easyconfigs of RAxML build one arbitrary binary of RAxML and create a symlink with the name raxmlHPC. This is a bad approach because the different binaries of RAxML have different requirements and features. For instance, binaries using MPI have a minimum number of trees to process

This is RAxML MPI Process Number: 0

WARNING: The number of threads is currently set to 0
You can specify the number of threads to run via -T numberOfThreads
NumberOfThreads must be set to an integer value greater than 1

RAxML, will now set the number of threads automatically to 2 !

Error: you are running the parallel MPI program but only want to compute one tree
For the MPI version you must specify a number of trees greater than 1 with the -# or -N option

lexming · 2020-09-16T17:50:58Z

Tests in easybuilders/easybuild-easyconfigs#11294

lexming · 2023-02-27T09:45:17Z

Test reports

Auto-detection
Custom optarch

lexming · 2023-02-27T12:01:38Z

Test report by @lexming

Overview of tested easyconfigs (in order)

SUCCESS RAxML-8.2.12-GCC-11.3.0.eb
SUCCESS RAxML-8.2.12-gompi-2022a.eb

Build succeeded for 2 out of 2 (2 easyconfigs in total)
node300.hydra.os - Linux CentOS Linux 7.9.2009, x86_64, Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/c11b4d9e09219333dbd36fcf85a32ba3 for a full test report.

boegel · 2023-03-15T17:30:48Z

@lexming Some existing RAxML easyconfigs have raxmlHPC -v and raxmlHPC -h as sanity check commands, we can let this easyblock run those too?

boegel · 2023-03-16T09:37:05Z

+from easybuild.tools.systemtools import get_cpu_features, HAVE_ARCHSPEC
+from easybuild.tools.toolchain.compiler import OPTARCH_GENERIC
+
+RAXML_BINARY_NAME = "raxmlHPC"


Not sure RAXML_BINARY_NAME is useful, since it's only used in one place currently?

boegel · 2023-03-16T09:37:23Z

+
+RAXML_BINARY_NAME = "raxmlHPC"
+# Supported instruction sets ordered by priority (high to low) and related CPU features
+RAXML_INSTRUCT_SETS = ["AVX2", "AVX", "SSE3"]


What about AVX-512? If there's a reason no to include that, add a comment on it?
Where can the list of supported instruction sets be found (docs link? README?)

Also, out of curiosity, does it also support NEON (for aarch64)?
If someone cares enough the easyblock can be updated later to also cover that, doesn't need to be done here.

They don't support that in their makefiles, this choice reflects what they actually set in CFLAGS (they do not respect the CFLAGS in the environment)

boegel · 2023-03-16T09:38:40Z

+# Supported instruction sets ordered by priority (high to low) and related CPU features
+RAXML_INSTRUCT_SETS = ["AVX2", "AVX", "SSE3"]
+RAXML_CPU_FEATURES = {
+    "SSE3": ["sse3", "see4_1", "sse4_2"],


typo:

Suggested change

"SSE3": ["sse3", "see4_1", "sse4_2"],

"SSE3": ["sse3", "sse4_1", "sse4_2"],

boegel · 2023-03-16T09:40:17Z

+        """RAxML easyblock constructor, define class variables."""
+        super(EB_RAxML, self).__init__(*args, **kwargs)
+
+        def filter_optarch_features(support_features):


Make this a function outside of the easyblock? Same for others below.

Advantage is that tests can be added easily to CI for these, and code becomes quite a bit easier to parse for humans

boegel · 2023-03-16T09:43:47Z

+                return feature.lower() in host
+
+            try:
+                return any(f in get_cpu_features() for f in RAXML_CPU_FEATURES[feature])


Would be nicer with a single return as the end, I'm not a big fan of inline returns, see also https://realpython.com/python-return-statement/#remembering-the-return-value (but that's partially personal, I guess)

Maybe this function should be moved into framework, it's pretty generic... Same for filter_optarch_features

boegel · 2023-03-16T09:45:41Z

+            # unrestricted optimization settings, set optimization level for host micro-architecture
+            cpu_features = [feat for feat in cpu_features if has_cpu_feature(feat)]
+            dbg_msg = "Enabling the following CPU optimizations for RAxML by autodetection: %s"
+            self.log.debug(dbg_msg, ", ".join(cpu_features))


Make this info log message?

boegel · 2023-03-16T09:46:12Z

+            dbg_msg = "Enabling the following CPU optimizations for RAxML by autodetection: %s"
+            self.log.debug(dbg_msg, ", ".join(cpu_features))
+            # add generic build
+            cpu_features.append(None)


Can you clarify (in the comment) why this is done?

boegel · 2023-03-16T09:46:46Z

+            parallel_features.extend(RAXML_PARALLEL_FEATURES["mpi"])
+
+        # List of builds to carry out
+        self.target_makefiles = list_filename_variants(cpu_features, parallel_features, "Makefile", "gcc", ".")


Is always using gcc OK? What if we're installing RAxML with Intel compilers?

boegel · 2023-03-16T09:48:04Z

+            if not any(feature in mf for feature in RAXML_PARALLEL_FEATURES["mpi"]):
+                cc_opt = compiler_nompi
+            self.cfg["buildopts"] = '-f %s CC="%s" %s' % (mf, cc_opt, user_buildopts)
+            self.log.debug("Building RAxML makefile with %s: %s", cc_opt, mf)


log.info?

boegel · 2023-03-16T09:49:02Z

+            "files": [os.path.join("bin", x) for x in self.target_bins],
+            "dirs": ["share/manual", "share/usefulScripts"],
+        }
+        super(EB_RAxML, self).sanity_check_step(custom_paths=custom_paths)


Also include a custom sanity check command like raxmlHPC -h?

akesandgren · 2023-10-23T08:52:26Z

@lexming this easyblock needs some attention...

ocaisa · 2025-10-08T09:19:19Z

There's good knowledge in here about what to actually build (it's clear you probably need a serial and a parallel build). The actual Makefiles they ship are not very complicated. Rather than do architecture detection etc., what I'm thinking of doing is just creating patched versions that respect the EasyBuild compilers and options, build both the serial and parallel versions and leave it at that.

The files that are being compile are 10+ years old, and in the commit that includes the performance optimisation they mention the improvement being 5-10% with GCC 4.7+...how would that compare with a modern compiler?

boegel · 2025-10-08T11:29:53Z

Just to have a cross-reference: see also:

use generic version of RAxML v8.2.13 as dependency for bcgTree and Parsnp easybuild-easyconfigs#23975

lexming · 2025-11-18T08:54:17Z

@ocaisa I fully agree, closing this in favour of easybuilders/easybuild-easyconfigs#23975

add easyblock for RAxML

0e8b180

lexming added the new label Sep 16, 2020

lexming added this to the 4.x milestone Sep 16, 2020

remove easybuild.tools.environment from raxml

5b40659

lexming mentioned this pull request Sep 16, 2020

{bio}[GCC/9.3.0,gompi/2020a] RAxML v8.2.12 easybuilders/easybuild-easyconfigs#11294

Closed

boegel requested changes Sep 22, 2020

View reviewed changes

Comment thread easybuild/easyblocks/r/raxml.py Outdated

Comment thread easybuild/easyblocks/r/raxml.py Outdated

Comment thread easybuild/easyblocks/r/raxml.py Outdated

lexming added 4 commits September 23, 2020 17:09

fix formatting of raxml.py

f7afebd

Merge branch 'develop' into raxml

e2626a8

refactor CPU features detection and build variants in RAxML

71d922e

follow optarch settings in RAxML easyblock

e5f5114

lexming mentioned this pull request Feb 27, 2023

{bio}[GCC/11.3.0,gompi/2022a] RAxML v8.2.12 easybuilders/easybuild-easyconfigs#17433

Closed

lexming added 2 commits February 27, 2023 01:05

fix optarch fallback chain in RAxML easyblock

8b33807

consider boolean optarch settings from toolchianopts in RAxML easyblock

7466ce8

boegel modified the milestones: 4.x, next release (4.7.1) Mar 1, 2023

boegel reviewed Mar 16, 2023

View reviewed changes

boegel modified the milestones: next release (4.7.1), release after 4.7.1 Mar 16, 2023

boegel modified the milestones: next release (4.7.2), release after 4.7.2 May 10, 2023

boegel modified the milestones: 4.7.3, release after 4.7.3 Jul 5, 2023

boegel modified the milestones: next release (4.8.1?), release after 4.8.1 Sep 3, 2023

boegel removed this from the next release (4.8.2) milestone Oct 27, 2023

boegel added this to the release after 4.8.2 milestone Oct 27, 2023

boegel modified the milestones: next release (4.9.0), release after 4.9.0 Dec 26, 2023

boegel modified the milestones: 4.9.1, release after 4.9.1 Apr 3, 2024

boegel modified the milestones: 4.9.2, release after 4.9.2 Jun 6, 2024

boegel modified the milestones: 4.9.3, release after 4.9.3 Sep 11, 2024

boegel modified the milestones: release after 4.9.4, release after 5.0.0 Mar 18, 2025

boegel modified the milestones: next release (5.1.0), 5.x May 22, 2025

boegel mentioned this pull request Oct 8, 2025

use generic version of RAxML v8.2.13 as dependency for bcgTree and Parsnp easybuilders/easybuild-easyconfigs#23975

Merged

lexming closed this Nov 18, 2025

lexming deleted the raxml branch November 18, 2025 08:54

lexming modified the milestones: 5.x, next release (5.2.0?) Nov 18, 2025

	"SSE3": ["sse3", "see4_1", "sse4_2"],
	"SSE3": ["sse3", "sse4_1", "sse4_2"],

Conversation

lexming commented Sep 16, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lexming commented Sep 16, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lexming commented Feb 27, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Test reports

Uh oh!

lexming commented Feb 27, 2023

Overview of tested easyconfigs (in order)

Uh oh!

boegel commented Mar 15, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akesandgren commented Oct 23, 2023

Uh oh!

ocaisa commented Oct 8, 2025

Uh oh!

boegel commented Oct 8, 2025

Uh oh!

lexming commented Nov 18, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

lexming commented Sep 16, 2020 •

edited

Loading

lexming commented Feb 27, 2023 •

edited

Loading