NetCDF: Filter error: undefined filter encountered

[minomicetto]

# Platform specification
Architecture: aarch64
NetCDF C Library: 4.9.2, but the same problem is raised even for 4.9.0
Operating System: Linux CentOS 9 Stream - Linux localhost.localdomain 5.14.0-340.el9.aarch64 #1 SMP PREEMPT_DYNAMIC Mon Jul 17 12:31:16 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux

gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-redhat-linux/11/lto-wrapper
Target: aarch64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-gnu-indirect-function --build=aarch64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.1 20230605 (Red Hat 11.4.1-2) (GCC)

mpicc -v
mpicc for MPICH version 4.1.1
Using built-in specs.
Reading specs from /usr/lib/rpm/redhat/redhat-hardened-cc1
Reading specs from /usr/lib/rpm/redhat/redhat-annobin-cc1
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-redhat-linux/11/lto-wrapper
Target: aarch64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-gnu-indirect-function --build=aarch64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.1 20230605 (Red Hat 11.4.1-2) (GCC)

mpicxx -v
mpicxx for MPICH version 4.1.1
Using built-in specs.
Reading specs from /usr/lib/rpm/redhat/redhat-hardened-cc1
Reading specs from /usr/lib/rpm/redhat/redhat-annobin-cc1
COLLECT_GCC=g++
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/aarch64-redhat-linux/11/lto-wrapper
Target: aarch64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-host-pie --enable-host-bind-now --enable-languages=c,c++,fortran,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --enable-plugin --enable-initfini-array --without-isl --enable-multilib --with-linker-hash-style=gnu --enable-gnu-indirect-function --build=aarch64-redhat-linux --with-build-config=bootstrap-lto --enable-link-serialization=1
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 11.4.1 20230605 (Red Hat 11.4.1-2) (GCC)

The steps performed to install and to access a Zarr dataset via the library are listed below.

# Download and install s3 library
git clone https://github.com/aws/aws-sdk-cpp.git
cd aws-sdk-cpp
git submodule update --init --recursive
srcdir=$(pwd)
mkdir build
cd build
cmake $srcdir -DBUILD_ONLY=s3 -DCMAKE_BUILD_TYPE=Debug -DCMAKE_PREFIX_PATH=/path/to/aws-sdk-cpp
make -j2 -k
make install

# Download and install NetCDF library
wget https://downloads.unidata.ucar.edu/netcdf-c/4.9.2/netcdf-c-4.9.2.zip
unzip netcdf-c-4.9.2.zip
cd netcdf-c-4.9.2
./configure --prefix=/path/to/netcdf-4.9.2 --enable-s3 LDFLAGS="-L/path/to/aws-sdk-cpp/lib64 -laws-cpp-sdk-s3" CPPFLAGS=-I/path/to/aws-sdk-cpp/include CC=mpicc CPP=mpicxx
make -j2 -k
make install

# Download the input Zarr data
cd /data/zarr
wget -r -H -N --cut-dirs=4 --include-directories="/v1/" "https://swiftbrowser.dkrz.de/public/dkrz_0b2a0dcc-1430-4a8a-9f25-a6cb8924d92b/CMCC_CMIP/CMIP6.CMIP.CMCC.CMCC-CM2-HR4.historical.r1i1p1f1.day.pr.gn.v20200904/?show_all"

The tests are listed below:

# Tests
export LD_LIBRARY_PATH=/path/to/netcdf-4.9.2/lib:/path/to/aws-sdk-cpp/lib64:$LD_LIBRARY_PATH

## Test 1: passed
/path/to/netcdf-4.9.2/bin/ncdump -h s3://storage.googleapis.com/cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/highresSST-present/r1i1p1f1/Amon/rsds/gn/v20170706/#mode=zarr
netcdf #mode=zarr {
dimensions:
lat = 192 ;
bnds = 2 ;
lon = 288 ;
time = 804 ;
variables:
double lat(lat) ;
lat:axis = "Y" ;
lat:bounds = "lat_bnds" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:axis = "X" ;
lon:bounds = "lon_bnds" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
lon:units = "degrees_east" ;
double lon_bnds(lon, bnds) ;
float rsds(time, lat, lon) ;
rsds:cell_measures = "area: areacella" ;
rsds:cell_methods = "area: time: mean" ;
rsds:comment = "surface solar irradiance for UV calculations" ;
rsds:history = "2017-07-02T15:44:13Z altered by CMOR: Converted units from ' W m-2' to 'W m-2'." ;
rsds:long_name = "Surface Downwelling Shortwave Radiation" ;
rsds:original_units = "W m-2" ;
rsds:standard_name = "surface_downwelling_shortwave_flux_in_air" ;
rsds:units = "W m-2" ;
int64 time(time) ;
time:axis = "T" ;
time:bounds = "time_bnds" ;
time:calendar = "noleap" ;
time:long_name = "time" ;
time:standard_name = "time" ;
time:units = "hours since 1948-01-16 12:00:00.000000" ;
double time_bnds(time, bnds) ;
time_bnds:calendar = "365_day" ;
time_bnds:units = "days since 1948-01-01" ;

// global attributes:
:CDI = "Climate Data Interface version 1.8.0 (http://mpimet.mpg.de/cdi)" ;
:CDO = "Climate Data Operators version 1.8.0 (http://mpimet.mpg.de/cdo)" ;
:Conventions = "CF-1.7 CMIP-6.0" ;
:activity_id = "HighResMIP" ;
:cmor_version = "3.2.3" ;
:coordinates = "lon_bnds time_bnds lat_bnds" ;
:creation_date = "2017-07-02T15:44:13Z" ;
:data_specs_version = "01.00.23" ;
:end_year = "2014" ;
:experiment = "forced atmosphere experiment for 1950-2014" ;
:experiment_id = "highresSST-present" ;
:external_variables = "areacella" ;
:forcing_index = 1 ;
:frequency = "mon" ;
:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.highresSST-present.none.r1i1p1f1" ;
:grid = "native atmosphere regular grid (192x288 latxlon)" ;
:grid_label = "gn" ;
:history = "2017-07-02T15:44:13Z CMOR rewrote data to be consistent with CF standards and CMIP6 requirements.; 2019-04-23T15:50:59Z DataSpecsVersionAdd, FurtherInfoUrlToHttps" ;
:initialization_index = 1 ;
:institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ;
:institution_id = "CMCC" ;
:license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:min_number_yrs_per_sim = "65" ;
:mip_era = "CMIP6" ;
:nominal_resolution = "1x1 degree" ;
:parent_activity_id = "no parent" ;
:parent_experiment_id = "no parent" ;
:parent_mip_era = "no parent" ;
:physics_index = 1 ;
:product = "model-output" ;
:realization_index = 1 ;
:realm = "atmos" ;
:source = "CMCC-CM2-HR4 (2016): aerosol: prescribed MACv2-SP atmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa) atmosChem: none land: CLM4.5 (SP mode) landIce: none ocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m) ocnBgchem: none seaIce: CICE4.0" ;
:source_id = "CMCC-CM2-HR4" ;
:source_type = "AGCM" ;
:start_year = "1950" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "Amon" ;
:table_info = "Creation Date:(22 April 2017) MD5:d3f61e4124f67845cd50ff647ba0c673" ;
:tier = "1" ;
:title = "CMCC-CM2-HR4 model output prepared for CMIP6" ;
:tracking_id = "hdl:21.14100/6bc46076-c6c8-4f5b-acf0-d79baab75d7e" ;
:variable_id = "rsds" ;
:variant_label = "r1i1p1f1" ;
:status = "2019-11-16;created;by nhn2@columbia.edu" ;
}

## Test 2: failed
/path/to/netcdf-4.9.2/bin/ncdump -hs s3://storage.googleapis.com/cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/highresSST-present/r1i1p1f1/Amon/rsds/gn/v20170706/#mode=zarr
netcdf #mode=zarr {
dimensions:
lat = 192 ;
bnds = 2 ;
lon = 288 ;
time = 804 ;
variables:
double lat(lat) ;
lat:axis = "Y" ;
lat:bounds = "lat_bnds" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
lat:_Storage = "chunked" ;
lat:_ChunkSizes = 192 ;
NetCDF: Filter error: undefined filter encountered
Location: file ncdump.c; fcn pr_att_specials line 1060

## Test 3: failed
/path/to/netcdf-4.9.2/bin/ncdump -v lat s3://storage.googleapis.com/cmip6/CMIP6/HighResMIP/CMCC/CMCC-CM2-HR4/highresSST-present/r1i1p1f1/Amon/rsds/gn/v20170706/#mode=zarr
netcdf #mode=zarr {
dimensions:
lat = 192 ;
bnds = 2 ;
lon = 288 ;
time = 804 ;
variables:
double lat(lat) ;
lat:axis = "Y" ;
lat:bounds = "lat_bnds" ;
lat:long_name = "latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:axis = "X" ;
lon:bounds = "lon_bnds" ;
lon:long_name = "longitude" ;
lon:standard_name = "longitude" ;
lon:units = "degrees_east" ;
double lon_bnds(lon, bnds) ;
float rsds(time, lat, lon) ;
rsds:cell_measures = "area: areacella" ;
rsds:cell_methods = "area: time: mean" ;
rsds:comment = "surface solar irradiance for UV calculations" ;
rsds:history = "2017-07-02T15:44:13Z altered by CMOR: Converted units from ' W m-2' to 'W m-2'." ;
rsds:long_name = "Surface Downwelling Shortwave Radiation" ;
rsds:original_units = "W m-2" ;
rsds:standard_name = "surface_downwelling_shortwave_flux_in_air" ;
rsds:units = "W m-2" ;
int64 time(time) ;
time:axis = "T" ;
time:bounds = "time_bnds" ;
time:calendar = "noleap" ;
time:long_name = "time" ;
time:standard_name = "time" ;
time:units = "hours since 1948-01-16 12:00:00.000000" ;
double time_bnds(time, bnds) ;
time_bnds:calendar = "365_day" ;
time_bnds:units = "days since 1948-01-01" ;

// global attributes:
:CDI = "Climate Data Interface version 1.8.0 (http://mpimet.mpg.de/cdi)" ;
:CDO = "Climate Data Operators version 1.8.0 (http://mpimet.mpg.de/cdo)" ;
:Conventions = "CF-1.7 CMIP-6.0" ;
:activity_id = "HighResMIP" ;
:cmor_version = "3.2.3" ;
:coordinates = "lon_bnds time_bnds lat_bnds" ;
:creation_date = "2017-07-02T15:44:13Z" ;
:data_specs_version = "01.00.23" ;
:end_year = "2014" ;
:experiment = "forced atmosphere experiment for 1950-2014" ;
:experiment_id = "highresSST-present" ;
:external_variables = "areacella" ;
:forcing_index = 1 ;
:frequency = "mon" ;
:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.highresSST-present.none.r1i1p1f1" ;
:grid = "native atmosphere regular grid (192x288 latxlon)" ;
:grid_label = "gn" ;
:history = "2017-07-02T15:44:13Z CMOR rewrote data to be consistent with CF standards and CMIP6 requirements.; 2019-04-23T15:50:59Z DataSpecsVersionAdd, FurtherInfoUrlToHttps" ;
:initialization_index = 1 ;
:institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ;
:institution_id = "CMCC" ;
:license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file). The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:min_number_yrs_per_sim = "65" ;
:mip_era = "CMIP6" ;
:nominal_resolution = "1x1 degree" ;
:parent_activity_id = "no parent" ;
:parent_experiment_id = "no parent" ;
:parent_mip_era = "no parent" ;
:physics_index = 1 ;
:product = "model-output" ;
:realization_index = 1 ;
:realm = "atmos" ;
:source = "CMCC-CM2-HR4 (2016): aerosol: prescribed MACv2-SP atmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa) atmosChem: none land: CLM4.5 (SP mode) landIce: none ocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m) ocnBgchem: none seaIce: CICE4.0" ;
:source_id = "CMCC-CM2-HR4" ;
:source_type = "AGCM" ;
:start_year = "1950" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "Amon" ;
:table_info = "Creation Date:(22 April 2017) MD5:d3f61e4124f67845cd50ff647ba0c673" ;
:tier = "1" ;
:title = "CMCC-CM2-HR4 model output prepared for CMIP6" ;
:tracking_id = "hdl:21.14100/6bc46076-c6c8-4f5b-acf0-d79baab75d7e" ;
:variable_id = "rsds" ;
:variant_label = "r1i1p1f1" ;
:status = "2019-11-16;created;by nhn2@columbia.edu" ;
data:

NetCDF: Filter error: undefined filter encountered
Location: file vardata.c; fcn print_rows line 478

## Test 4: passed
/path/to/netcdf-4.9.2/bin/ncdump -h file:///data/zarr/swift.dkrz.de#mode=zarr
netcdf swift.dkrz {
dimensions:
lat = 192 ;
bnds = 2 ;
lon = 288 ;
time = 60225 ;
variables:
double lat(lat) ;
lat:axis = "Y" ;
lat:bounds = "lat_bnds" ;
lat:long_name = "Latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:axis = "X" ;
lon:bounds = "lon_bnds" ;
lon:long_name = "Longitude" ;
lon:standard_name = "longitude" ;
lon:units = "degrees_east" ;
double lon_bnds(lon, bnds) ;
float pr(time, lat, lon) ;
pr:cell_measures = "area: areacella" ;
pr:cell_methods = "area: time: mean" ;
pr:comment = "includes both liquid and solid phases" ;
pr:long_name = "Precipitation" ;
pr:missing_value = 1.00000002004088e+20 ;
pr:original_name = "PRECT" ;
pr:standard_name = "precipitation_flux" ;
pr:units = "kg m-2 s-1" ;
int64 time(time) ;
time:axis = "T" ;
time:bounds = "time_bnds" ;
time:calendar = "noleap" ;
time:long_name = "time" ;
time:standard_name = "time" ;
time:units = "days since 1850-01-01 12:00:00.000000" ;
double time_bnds(time, bnds) ;
time_bnds:calendar = "365_day" ;
time_bnds:units = "days since 1850-01-01" ;

// global attributes:
:Conventions = "CF-1.7 CMIP-6.2" ;
:activity_id = "CMIP" ;
:branch_method = "standard" ;
:branch_time_in_child = 0. ;
:branch_time_in_parent = 0. ;
:cmor_version = "3.5.0" ;
:comment = "none" ;
:contact = "T. Lovato" ;
:data_specs_version = "01.00.31" ;
:experiment = "all-forcing simulation of the recent past" ;
:experiment_id = "historical" ;
:external_variables = "areacella" ;
:forcing_index = 1 ;
:frequency = "day" ;
:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.historical.none.r1i1p1f1" ;
:grid = "native atmosphere regular grid" ;
:grid_label = "gn" ;
:history = "Converted and written to swift cloud with tzis version 0.1" ;
:initialization_index = 1 ;
:institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ;
:institution_id = "CMCC" ;
:license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:mip_era = "CMIP6" ;
:nominal_resolution = "100 km" ;
:parent_activity_id = "CMIP" ;
:parent_experiment_id = "piControl" ;
:parent_mip_era = "CMIP6" ;
:parent_source_id = "CMCC-CM2-HR4" ;
:parent_time_units = "days since 1850-01-01" ;
:parent_variant_label = "r1i1p1f1" ;
:physics_index = 1 ;
:product = "model-output" ;
:realization_index = 1 ;
:realm = "atmos" ;
:references = "none" ;
:run_variant = "1st realization" ;
:source = "CMCC-CM2-HR4 (2016): \naerosol: prescribed MACv2-SP\natmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa)\natmosChem: none\nland: CLM4.5 (SP mode)\nlandIce: none\nocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m)\nocnBgchem: none\nseaIce: CICE4.0" ;
:source_id = "CMCC-CM2-HR4" ;
:source_type = "AOGCM" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "day" ;
:table_info = "Creation Date:(05 February 2020) MD5:6a248fd76c55aa6d6f7b3cc6866b5faf" ;
:title = "CMCC-CM2-HR4 output prepared for CMIP6" ;
:tracking_id = "7af29565-027f-4d81-a74b-bee9a1235c0d" ;
:variable_id = "pr" ;
:variant_label = "r1i1p1f1" ;
}

## Test 5: failed
/path/to/netcdf-4.9.2/bin/ncdump -hs file:///data/zarr/swift.dkrz.de#mode=zarr
netcdf swift.dkrz {
dimensions:
lat = 192 ;
bnds = 2 ;
lon = 288 ;
time = 60225 ;
variables:
double lat(lat) ;
lat:axis = "Y" ;
lat:bounds = "lat_bnds" ;
lat:long_name = "Latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
lat:_Storage = "chunked" ;
lat:_ChunkSizes = 192 ;
NetCDF: Filter error: undefined filter encountered
Location: file ncdump.c; fcn pr_att_specials line 1060

## Test 6: failed
/path/to/netcdf-4.9.2/bin/ncdump -v lat file:///data/zarr/swift.dkrz.de#mode=zarr
netcdf swift.dkrz {
dimensions:
lat = 192 ;
bnds = 2 ;
lon = 288 ;
time = 60225 ;
variables:
double lat(lat) ;
lat:axis = "Y" ;
lat:bounds = "lat_bnds" ;
lat:long_name = "Latitude" ;
lat:standard_name = "latitude" ;
lat:units = "degrees_north" ;
double lat_bnds(lat, bnds) ;
double lon(lon) ;
lon:axis = "X" ;
lon:bounds = "lon_bnds" ;
lon:long_name = "Longitude" ;
lon:standard_name = "longitude" ;
lon:units = "degrees_east" ;
double lon_bnds(lon, bnds) ;
float pr(time, lat, lon) ;
pr:cell_measures = "area: areacella" ;
pr:cell_methods = "area: time: mean" ;
pr:comment = "includes both liquid and solid phases" ;
pr:long_name = "Precipitation" ;
pr:missing_value = 1.00000002004088e+20 ;
pr:original_name = "PRECT" ;
pr:standard_name = "precipitation_flux" ;
pr:units = "kg m-2 s-1" ;
int64 time(time) ;
time:axis = "T" ;
time:bounds = "time_bnds" ;
time:calendar = "noleap" ;
time:long_name = "time" ;
time:standard_name = "time" ;
time:units = "days since 1850-01-01 12:00:00.000000" ;
double time_bnds(time, bnds) ;
time_bnds:calendar = "365_day" ;
time_bnds:units = "days since 1850-01-01" ;

// global attributes:
:Conventions = "CF-1.7 CMIP-6.2" ;
:activity_id = "CMIP" ;
:branch_method = "standard" ;
:branch_time_in_child = 0. ;
:branch_time_in_parent = 0. ;
:cmor_version = "3.5.0" ;
:comment = "none" ;
:contact = "T. Lovato" ;
:data_specs_version = "01.00.31" ;
:experiment = "all-forcing simulation of the recent past" ;
:experiment_id = "historical" ;
:external_variables = "areacella" ;
:forcing_index = 1 ;
:frequency = "day" ;
:further_info_url = "https://furtherinfo.es-doc.org/CMIP6.CMCC.CMCC-CM2-HR4.historical.none.r1i1p1f1" ;
:grid = "native atmosphere regular grid" ;
:grid_label = "gn" ;
:history = "Converted and written to swift cloud with tzis version 0.1" ;
:initialization_index = 1 ;
:institution = "Fondazione Centro Euro-Mediterraneo sui Cambiamenti Climatici, Lecce 73100, Italy" ;
:institution_id = "CMCC" ;
:license = "CMIP6 model data produced by CMCC is licensed under a Creative Commons Attribution ShareAlike 4.0 International License (https://creativecommons.org/licenses). Consult https://pcmdi.llnl.gov/CMIP6/TermsOfUse for terms of use governing CMIP6 output, including citation requirements and proper acknowledgment. Further information about this data, including some limitations, can be found via the further_info_url (recorded as a global attribute in this file) and at https:///pcmdi.llnl.gov/. The data producers and data providers make no warranty, either express or implied, including, but not limited to, warranties of merchantability and fitness for a particular purpose. All liabilities arising from the supply of the information (including any liability arising in negligence) are excluded to the fullest extent permitted by law." ;
:mip_era = "CMIP6" ;
:nominal_resolution = "100 km" ;
:parent_activity_id = "CMIP" ;
:parent_experiment_id = "piControl" ;
:parent_mip_era = "CMIP6" ;
:parent_source_id = "CMCC-CM2-HR4" ;
:parent_time_units = "days since 1850-01-01" ;
:parent_variant_label = "r1i1p1f1" ;
:physics_index = 1 ;
:product = "model-output" ;
:realization_index = 1 ;
:realm = "atmos" ;
:references = "none" ;
:run_variant = "1st realization" ;
:source = "CMCC-CM2-HR4 (2016): \naerosol: prescribed MACv2-SP\natmos: CAM4 (1deg; 288 x 192 longitude/latitude; 26 levels; top at ~2 hPa)\natmosChem: none\nland: CLM4.5 (SP mode)\nlandIce: none\nocean: NEMO3.6 (ORCA0.25 1/4 deg from the Equator degrading at the poles; 1442 x 1051 longitude/latitude; 50 vertical levels; top grid cell 0-1 m)\nocnBgchem: none\nseaIce: CICE4.0" ;
:source_id = "CMCC-CM2-HR4" ;
:source_type = "AOGCM" ;
:sub_experiment = "none" ;
:sub_experiment_id = "none" ;
:table_id = "day" ;
:table_info = "Creation Date:(05 February 2020) MD5:6a248fd76c55aa6d6f7b3cc6866b5faf" ;
:title = "CMCC-CM2-HR4 output prepared for CMIP6" ;
:tracking_id = "7af29565-027f-4d81-a74b-bee9a1235c0d" ;
:variable_id = "pr" ;
:variant_label = "r1i1p1f1" ;
data:

NetCDF: Filter error: undefined filter encountered
Location: file vardata.c; fcn print_rows line 478
lat =

I think the errors raised in tests 2, 3, 5 and 6 are due to the same cause.
Could you help me to solve this issue?

[DennisHeimbigner]

First, what configure options (or CMake options) did you use?
Second, what value did you use for HDF5_PLUGIN_PATH?

[minomicetto]

I built the library by appending the option --with-plugin-dir as follows:

./configure --prefix=/path/to/netcdf-4.9.2 --enable-s3 --with-plugin-dir=/path/to/netcdf-4.9.2/lib/plugins LDFLAGS="-L/path/to/aws-sdk-cpp/lib64 -laws-cpp-sdk-s3" CPPFLAGS=-I/path/to/aws-sdk-cpp/include CC=mpicc CPP=mpicxx

This resulted that in /path/to/netcdf-4.9.2/lib/plugins I can find the related libraries:

ls /path/to/netcdf-4.9.2/lib/plugins
lib__nch5blosc.la lib__nch5bzip2.so.0 lib__nch5fletcher32.la lib__nch5shuffle.so.0 lib__nch5zstd.la lib__nczhdf5filters.so.0
lib__nch5blosc.so lib__nch5bzip2.so.0.0.0 lib__nch5fletcher32.so lib__nch5shuffle.so.0.0.0 lib__nch5zstd.so lib__nczhdf5filters.so.0.0.0
lib__nch5blosc.so.0 lib__nch5deflate.la lib__nch5fletcher32.so.0 lib__nch5szip.la lib__nch5zstd.so.0 lib__nczstdfilters.la
lib__nch5blosc.so.0.0.0 lib__nch5deflate.so lib__nch5fletcher32.so.0.0.0 lib__nch5szip.so lib__nch5zstd.so.0.0.0 lib__nczstdfilters.so
lib__nch5bzip2.la lib__nch5deflate.so.0 lib__nch5shuffle.la lib__nch5szip.so.0 lib__nczhdf5filters.la lib__nczstdfilters.so.0
lib__nch5bzip2.so lib__nch5deflate.so.0.0.0 lib__nch5shuffle.so lib__nch5szip.so.0.0.0 lib__nczhdf5filters.so lib__nczstdfilters.so.0.0.0

Then, I set HDF5_PLUGIN_PATH to /path/to/netcdf-4.9.2/lib/plugins as follows:

export HDF5_PLUGIN_PATH=/path/to/netcdf-4.9.2/lib/plugin

and now all the tests passed.
I thank you.

[minomicetto]

Now, I am testing my application using the NetCDF library and... the same error is raised.
However, in this case the problem is not due to HDF5_PLUGIN_PATH as the env variable is correctly set and most of above plugins are loaded correctly (all the plugins named lib__nch5*). In particular, by appending a simple print of dlerror() at https://github.com/Unidata/netcdf-c/blob/main/libncpoco/cp_unix.c#L137 I obtained that

dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczhdf5filters.so.0: undefined symbol: nc_inq_var_endian
dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczhdf5filters.so: undefined symbol: nc_inq_var_endian
dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so.0: undefined symbol: nc_inq_type
dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so: undefined symbol: nc_inq_type
dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczhdf5filters.so.0.0.0: undefined symbol: nc_inq_var_endian
dlopen(): /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so.0.0.0: undefined symbol: nc_inq_type

The problem occurs even if LD_LIBRARY_PATH includes /path/to/netcdf-4.9.2/lib/plugins

I have obtained a similar problem even over another architecture

Linux n179 3.10.0-957.el7.x86_64 #1 SMP Thu Nov 8 23:39:32 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

but, in this case, already the execution of dopen() results in error as follows:

symbol lookup error: /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so.0: undefined symbol: nc_inq_var

Could you help me to solve this issue?

[WardF]

I'll take a look, I'm curious if the LD_LIBRARY_PATH also contains the path to libnetcdf; it is complaining about undefined symbols that should be found in libnetcdf.

[DennisHeimbigner]

Not all filters need to use libnetcdf, so check the Makefile to see if this one has it.

[minomicetto]

@WardF: my LD_LIBRARY_PATH already includes libnetcdf as I am testing an application that uses it.

LD_LIBRARY_PATH=/path/to/hdf5-1.14.1-2/lib:/path/to/netcdf-4.9.2/lib:/path/to/aws-sdk-cpp/lib64

To this I have appended even the reference to /path/to/netcdf-4.9.2/lib/plugins, however without resulting in success.

@DennisHeimbigner: I am going to check the Makefile of the plugins and to refer you, but I have found no solution so far. Let me say if you suggest something to do, please.

[minomicetto]

I have solved simply by forcing the library to link itself. Maybe, this affects only the intended plugins, as the reflexive link does not impact the building phase (while the library does not exist yet).
In pratice, I have used the following configuration:

./configure --prefix=/path/to/netcdf-4.9.2 --enable-s3 LDFLAGS="-L/path/to/aws-sdk-cpp/lib64 -laws-cpp-sdk-s3 -L/path/to/netcdf-4.9.2/lib -lnetcdf" CPPFLAGS=-I/path/to/aws-sdk-cpp/include CC=mpicc CPP=mpicxx

I think there is a more smart approach to solve the problem but, in the meantime, we can close this issue.

[DennisHeimbigner]

Try adding directory containing libnetcdf to LD_LIBRARY_PATH

[minomicetto]

@DennisHeimbigner: as I replied before, LD_LIBRARY_PATH has always included the path to libnetcdf in my tests; in fact, my application exploits it and they are regularly linked:

$ ldd /path/to/my/application
linux-vdso.so.1 (0x0000ffffa4d41000)
libnetcdf.so.19 => /path/to/libnetcdf.so.19 (0x0000ffffa4a00000)
libltdl.so.7 => /lib64/libltdl.so.7 (0x0000ffffa4ce3000)
libm.so.6 => /lib64/libm.so.6 (0x0000ffffa4c42000)
libc.so.6 => /lib64/libc.so.6 (0x0000ffffa4852000)
[...]

The problem is the link between each ncz plugin and libnetcdf: even if the LD_LIBRARY_PATH includes libnetcdf (useless during the building), the plugins are built without an explicit reference to it. In fact, the ldd tool applied to a filter does not report the connection:

$ ldd /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so
linux-vdso.so.1 (0x0000ffff879d1000)
libc.so.6 => /lib64/libc.so.6 (0x0000ffff877c5000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff87994000)

I think that connection should rightly be established on loading the plugin, but this does not happend in my case.
Instead, adding the reference to libnetcdf to LDFLAGS, the filter includes an explicit connection even before loading it (and many other links, in effect):

$ ldd /path/to/netcdf-4.9.2/lib/plugins/lib__nczstdfilters.so
linux-vdso.so.1 (0x0000ffff98cbd000)
libnetcdf.so.19 => /path/to/libnetcdf.so.19 (0x0000ffff98a00000)
libc.so.6 => /lib64/libc.so.6 (0x0000ffff98852000)
/lib/ld-linux-aarch64.so.1 (0x0000ffff98c80000)
libstdc++.so.6 => /lib64/libstdc++.so.6 (0x0000ffff98600000)
libaws-cpp-sdk-s3.so => /path/to/libaws-cpp-sdk-s3.so (0x0000ffff98200000)
libhdf5_hl.so.310 => /path/to/libhdf5_hl.so.310 (0x0000ffff985ce000)
libhdf5.so.310 => /path/to/libhdf5.so.310 (0x0000ffff97c00000)
libm.so.6 => /lib64/libm.so.6 (0x0000ffff9852d000)
[...]

Then, I have no trouble to run the plugins.
Clearly, a smarter solution to link libnetcdf (only) to each ncz plugin, using default configuration, would be prefarable.
I thank you.

[DennisHeimbigner]

ok, please do ldd /path/to/netcdf-4.9.2/lib/plugins/lib__nczhdf5filters.so.0 to see where it thinks
libnetcdf is located.