NetCDF: HDF error when creating a lot of variables and attributes

[wkliao]

NetCDF 4.8.1
HDF5 1.13.0
MPICH 3.4.3
gcc 8.5.0

I encountered "NetCDF: HDF error" when running a parallel program that creates
an HDF5-based NetCDF4 file. The test program that can reproduce the error is
available in icase_def.c

The test program follows the E3SM I/O pattern by creating the a large number of
variables and attributes:
27 global attributes
21 dimensions
560 variables
and each variable has a few attributes.
The test program does not call any nc_put_var* APIs.

When using one MPI process, the test program ran fine.
But when running 4 MPI processes, it printed the following errors and hung
at nc_enddef.

mpiexec -n 4 ./icase_def
Error at icase_def.c:35 : NetCDF: HDF error
Error at icase_def.c:35 : NetCDF: HDF error
Error at icase_def.c:35 : NetCDF: HDF error

[wkliao]

Further investigation reveals that the location returns the error is at

netcdf-c/libhdf5/nc4hdf.c

Lines 1412 to 1413 in cd0f169

	if (H5DSattach_scale(hdf5_var->hdf_datasetid, dsid, d) < 0)
	return NC_EHDFERR;

In issue #1822, @brtnfld mentioned that "HDF5 does not test any of the HL APIs in a parallel setting". Given that, I wonder if netcdf plans to resolve the bug by taking @brtnfld's suggestion.

[DennisHeimbigner]

I wonder if this PR #2161
would fix the problem?

[wkliao]

Yes. Thanks.
I assume this PR will go to 4.9.0.