NC_MAX_NAME error on HDF4 file when names are within limits

[djhoese]

I am trying to read an HDF4 file with netcdf4-python 1.5.3 from conda-forge which uses libnetcdf 4.7.3 from conda-forge, but I've seen this in other versions to. Currently tested on an Ubuntu system and a CentOS system.

Example file: ftp://ftp.ssec.wisc.edu/pub/davidh/netcdf_debug/geocatL2.GOES-17.CONUS.2020038.190630.hdf

Running ncdump -h geocatL2.GOES-17.CONUS.2020038.190630.hdf produces (full path filtered out):

ncdump: geocatL2.GOES-17.CONUS.2020038.190630.hdf: geocatL2.GOES-17.CONUS.2020038.190630.hdf: NetCDF: NC_MAX_NAME exceeded

If I load the data with the pyhdf library and check the length of all variable names and attribute names they all seem under 100 and from what I can tell the limit in the NetCDF-C library should be 256. Any ideas?

[edwardhartnett]

The problem stems from this dimension:
byte_segmentsenterprise_v2_cldphase_7_10_11_13_14_15_Cldphase_Qpi
This is 65 chars long. As we see in the netcdf.h file:

/** This is the max size of an SD dataset name in HDF4 (from HDF4 documentation).*/
#define NC_MAX_HDF4_NAME 64

OK, maybe HDF4 has changed and can allow longer names. It turns out that NC_MAX_HDF4_NAME is also unnecessarily re-refined to the same value (64) in hdf4dispatch.h. I have removed that.

Looking at our HDF4 code, it's perfectly safe for any HDF4 name length that is less than NC_MAX_NAME. So I have raised NC_MAX_HDF4_NAME to NC_MAX_NAME, and this allows me to do an ncdump -h -s on your file, successfully.

I will put up a PR shortly.

Can you tell me why you are looking at GOES-17 data in an HDF4 file? Isn't all GOES-17 data produced in netCDF/HDF5? Why not use those files?

[djhoese]

Can you tell me why you are looking at GOES-17 data in an HDF4 file? Isn't all GOES-17 data produced in netCDF/HDF5? Why not use those files?

These are files created by the "geocat" software which is a semi-generic set of algorithms that can be applied to any satellite instrument (theoretically). The generic algorithm part is why you get some really long variable and dimension names as it is a combination of all the various parts (this algorithm + these options + this variable + etc). The software is old and still produces HDF4 although I believe newer developments produce NetCDF4 files.

Thank you for all your help.