Building netCDF with HDF5 1.12.0 vs 1.10.6

[afernandezody]

Hello,
I've been building netCDF with HDF5 (v1.10.6) for a while but decided to upgrade to v1.12.0 for no particular reason other than simply having a more recent version. The compilations of HDF5 for both versions are with autotools and include the flag --with-zlib. However, the new compilation (from source) of netCDF (with cmake) returns the error:

...
-- Found HDF5: /opt/hdf5/lib/libhdf5.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.12.0") found components: C HL
-- Found HDF5 libraries version 1.12.0
-- Using HDF5 include dir: /opt/hdf5/include
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Looking for H5Pget_fapl_mpio in hdf5
-- Looking for H5Pget_fapl_mpio in hdf5 - not found
-- Looking for H5Pset_all_coll_metadata_ops in hdf5
-- Looking for H5Pset_all_coll_metadata_ops in hdf5 - not found
-- Looking for H5Z_SZIP in hdf5
-- Looking for H5Z_SZIP in hdf5 - not found
-- Looking for H5Pset_libver_bounds in hdf5
-- Looking for H5Pset_libver_bounds in hdf5 - not found
-- Looking for H5free_memory in hdf5
-- Looking for H5free_memory in hdf5 - not found
-- Looking for H5allocate_memory in hdf5
-- Looking for H5allocate_memory in hdf5 - not found
-- Looking for H5resize_memory in hdf5
-- Looking for H5resize_memory in hdf5 - not found
-- Looking for H5Dread_chunk in hdf5
-- Looking for H5Dread_chunk in hdf5 - not found
-- Performing Test HAVE_HDF5_ZLIB
-- Performing Test HAVE_HDF5_ZLIB - Failed
CMake Error at CMakeLists.txt:778 (MESSAGE):
  HDF5 was built without zlib.  Rebuild HDF5 with zlib.

I have cleaned up and recompiled, but it always spits out the same error. Other than the HDF5 libraries having different numbers, I haven't noticed any outstanding difference. Thanks.

[WardF]

Which version of netcdf-c are you building? I'm trying to duplicate this issue. Also, can you provide the libhdf5.settings file generated for your hdf5 1.12.0 build?

[afernandezody]

I cloned netCDF from the Github repository. The HDF5 info reads:

            SUMMARY OF THE HDF5 CONFIGURATION
            =================================

General Information:
-------------------
                   HDF5 Version: 1.12.0
                  Configured on: Tue Aug 11 20:39:39 UTC 2020
                  Configured by: ubuntu@instance-20200811-1633
                    Host system: x86_64-unknown-linux-gnu
              Uname information: Linux instance-20200811-1633 5.4.0-1021-oracle #21-Ubuntu SMP Fri Jul 10 06:33:24 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
                       Byte sex: little-endian
             Installation point: /opt/hdf5

Compiling Options:
------------------
                     Build Mode: debug
              Debugging Symbols: yes
                        Asserts: yes
                      Profiling: no
             Optimization Level: debug

Linking Options:
----------------
                      Libraries: static, shared
  Statically Linked Executables:
                        LDFLAGS:
                     H5_LDFLAGS:
                     AM_LDFLAGS:
                Extra libraries: -lz -ldl -lm
                       Archiver: ar
                       AR_FLAGS: cr
                         Ranlib: ranlib

Languages:
----------
                              C: yes
                     C Compiler: /usr/bin/gcc
                       CPPFLAGS:
                    H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L   -UNDEBUG -DH5AC_DEBUG -DH5B2_DEBUG -DH5CX_DEBUG -DH5D_DEBUG -DH5F_DEBUG -DH5HL_DEBUG -DH5I_DEBUG -DH5O_DEBUG -DH5S_DEBUG -DH5ST_DEBUG -DH5T_DEBUG -DH5Z_DEBUG -DH5_DEBUG_API
                    AM_CPPFLAGS:
                        C Flags:
                     H5 C Flags:  -std=c99  -Wall -Wcast-qual -Wconversion -Wextra -Wfloat-equal -Wformat=2 -Winit-self -Winvalid-pch -Wmissing-include-dirs -Wno-c++-compat -Wno-format-nonliteral -Wshadow -Wundef -Wwrite-strings -pedantic -Wbad-function-cast -Wcast-align -Wdeclaration-after-statement -Wdisabled-optimization -Wmissing-declarations -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wredundant-decls -Wstrict-prototypes -Wswitch-enum -Wswitch-default -Wunused-macros -Wunsafe-loop-optimizations -Wlogical-op -Wlarger-than=2560 -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wnull-dereference -Wunused-const-variable -Wduplicated-cond -Whsa -Wnormalized -Walloc-zero -Walloca -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=2 -Wimplicit-fallthrough=5 -Wrestrict -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -Wattribute-alias=2 -Wmissing-profile -ftrapv -fno-common -g -fno-omit-frame-pointer -Wno-aggregate-return -Wno-inline -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-suggest-attribute=const -Wno-suggest-attribute=pure -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -Og
                     AM C Flags:
               Shared C Library: yes
               Static C Library: yes


                        Fortran: no

                            C++: no

                           Java: no


Features:
---------
                   Parallel HDF5: no
Parallel Filtered Dataset Writes: no
              Large Parallel I/O: no
              High-level library: yes
                Build HDF5 Tests: yes
                Build HDF5 Tools: yes
                    Threadsafety: no
             Default API mapping: v112
  With deprecated public symbols: yes
          I/O filters (external): deflate(zlib)
                             MPE: no
                   Map (H5M) API: no
                      Direct VFD: no
              (Read-Only) S3 VFD: no
            (Read-Only) HDFS VFD: no
                         dmalloc: no
  Packages w/ extra debug output: AC,B2,CX,D,F,HL,I,O,S,ST,T,Z
                     API tracing: yes
            Using memory checker: no
 Memory allocation sanity checks: yes
          Function stack tracing: no
       Strict file format checks: yes
    Optimization instrumentation: no

[afernandezody]

Were you able to reproduce the issue?

[WardF]

I was and am investigating now; sorry for the delayed response!

[edhartnett]

BTW I am building routinely with 1.12.0 in my CI, with no apparent problems.

…

On Fri, Aug 28, 2020 at 4:10 PM Ward Fisher ***@***.***> wrote: I was and am investigating now; sorry for the delayed response! — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#1806 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABCSXXBHRVYTMRD3W7SHSG3SDATNDANCNFSM4P3OYTKQ> .

[afernandezody]

I built the system with HDF5 1.10.6 (as I mentioned upgrading to 1.12.0 was a very minor thing). I will try again sometime in the future but maybe this was an isolated glitch.

[edwardhartnett]

Possibly this issue should be closed. I just checked and can confirm that hdf5-1.12.0 is running on all my branches and has been for some time, and has never failed where 1.10.6 has passed.

[WardF]

Agreed, I thought I had duplicated this but realized I had conflated it with a different HDF5 issue I had recreated. @afernandezody, if you can recreate this when you try again in the future, feel free to comment and/or reopen the issue. Thanks all!