Diagnose and reenable tests in el9 `bes_core` build

[hannahilea]

In #1288 we removed the --enable-dev flag from the docker build of bes_core. Doing this caused a flurry of test failures in the el9 build in the hdf5_handlerTest test suite. [1] In order to unblock other development work, we turned off tests for the el9 bes_core build.

We need to turn those tests back on, and fix any ensuing test failures. To do this, remove the if/else around testing here

bes/Dockerfile

Line 107 in 3ea3883

RUN if [ "$DIST" == "el9" ]; then \

For a faster development workflow, the bes_core dockerfile can be built locally.

---

The fact that this test is failing could be a canary that something else hdf5 is weird in the el9 build. or it could be a red herring.

Other thoughts:

• With the --enable-dev flag enabled when configuring/building the bes on el9 (within the bes_core docker image), make check succeeds.
• We do not know that these have ever succeeded on el9; it seems likely that they have never worked, and we just never knew it because we'd never tested them: Before we had the bes_core el9 docker build, we had the el9 rpm build, which did not run make check at all.
• On the other hand, for the rpms we were doing static linking---and in bes_core docker, we're doing dynamic linking. So that could be the issue here.
• On a third hand (borrowed from a friend), our el8 and OSX tests all pass, and are all using dynamic linking.
  • ...does that maybe point at an issue in hyrax-dependencies el9 build, maybe?
• Do the regression tests run on the hyrax-docker el9 hyrax image? That may provide more clues, assuming that there are hdf5 tests inside that test suite.

---

[1] At least the first of these failures seems to be related to hdf5 library linking. Example failure:

[bes_user@hyrax bes-testsuite]$ ./hdf5_handlerTest -v 13
## ------------------------------------------------------------------------------------ ##
## bes 3.21.1 test suite: bes.cfdmr.conf bes.default.conf besstandalone getdap getdap4. ##
## ------------------------------------------------------------------------------------ ##
13. hdf5_handlerTest.at:225: testing BESCMD $abs_srcdir/h5.cf/t_2d_2dll.nc4.h5.dds.bescmd ...
./hdf5_handlerTest.at:225: besstandalone -c $abs_builddir/bes.cfdmr.conf -i $abs_srcdir/h5.cf/t_2d_2dll.nc4.h5.dds.bescmd || true
stderr:
/home/bes_user/bes/standalone/.libs/lt-besstandalone: symbol lookup error: /home/bes_user/bes/modules/hdf5_handler/.libs/libhdf5_module.so: undefined symbol: _ZN6HDF5CF4File27Handle_General_NameClashingINS_9AttributeEEEvRSt3setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4lessIS9_ESaIS9_EERSt6vectorIPT_SaISH_EE
stdout:
./hdf5_handlerTest.at:225: diff -b -B $abs_srcdir/h5.cf/t_2d_2dll.nc4.h5.dds.bescmd.baseline stdout || diff -b -B $abs_srcdir/h5.cf/t_2d_2dll.nc4.h5.dds.bescmd.baseline stderr
stdout:
1,5d0
< Dataset {
<     Float32 mydata[xc = 4][yc = 4];
<     Float32 lat[xc = 4][yc = 4];
<     Float32 lon[xc = 4][yc = 4];
< } t_2d_2dll.nc4.h5;
1,5c1
< Dataset {
<     Float32 mydata[xc = 4][yc = 4];
<     Float32 lat[xc = 4][yc = 4];
<     Float32 lon[xc = 4][yc = 4];
< } t_2d_2dll.nc4.h5;
---
> /home/bes_user/bes/standalone/.libs/lt-besstandalone: symbol lookup error: /home/bes_user/bes/modules/hdf5_handler/.libs/libhdf5_module.so: undefined symbol: _ZN6HDF5CF4File27Handle_General_NameClashingINS_9AttributeEEEvRSt3setINSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt4lessIS9_ESaIS9_EERSt6vectorIPT_SaISH_EE
./hdf5_handlerTest.at:225: exit code was 1, expected 0
13. hdf5_handlerTest.at:225:  FAILED (hdf5_handlerTest.at:225)

## ------------- ##
## Test results. ##
## ------------- ##

ERROR: 1 test was run,
1 failed unexpectedly.

[jgallagher59701]

Here's a bit more info about this missing symbol. The function 'Handle_General_NameClashing' is a template function defined in the HDF5 handlers code in file HDF5CF.cc on line 1485. There are sibling functions defined in HDF5GMCF.cc and HDFEOS5CF.cc. It could be that the template function needs to be explicitly instantiated. A potential fix might be to move the definition of the template to a header. This is just me guessing, but we might throw this at codex and see what it says.