data values not right when opening Xarray-generated zarr with ncdump

[christine-e-smit]

ncdump version: 4.8.1 of Apr 25 2022 (this is what came with netCDF4-python 1.6.0 when I installed that)
xarray version: 2022.3.0
operating system: Mac OS Monterey
python version: 3.10.5

I found this issue using the netCDF4-python library, but I think it must be at the netcdf-c level because I'm seeing the same weird data values with ncdump. That's why I'm submitting the ticket here rather than in the netCDF-python project.

To reproduce the issue:

1. Create two directories: "download" for the netCDF file and "generated" for the zarr store.

2. Download the netcdf file into the download directory. Call it "GLDAS_NOAH025_3H.A20000101.0300.021.nc4": "https://hydro1.gesdisc.eosdis.nasa.gov/opendap/GLDAS/GLDAS_NOAH025_3H.2.1/2000/001/GLDAS_NOAH025_3H.A20000101.0300.021.nc4.nc4?time[0:1:0],lon[0:1:1439],lat[0:1:599],Rainf_tavg[0:1:0][0:1:599][0:1:1439]"

You will need to have earthdata login credentials download this URL. If you don't already have an earthdata login credential, you can create one for free here: https://urs.earthdata.nasa.gov/

3. Generate zarr using python's Xarray library. Verify that the latitude variable has values between -59.875 and 89.875

import xarray as xr
# read in the netCDF file
ds = xr.load_dataset("download/GLDAS_NOAH025_3H.A20000101.0300.021.nc4")

# check that latitude values look good. They should range from -59.875 to 89.875
print("Latitudes in original NetCDF4/HDF file:")
print(ds["lat"].values[:])

# write out zarr
ds.to_zarr("generated")

# read it in zarr using xarray
ds_from_zarr = xr.open_zarr("generated")

# print the latitudes again
print("Latitudes in zarr store, according to xarray:")
print(ds_from_zarr["lat"].values[:])

4. Now run ncdump on that zarr store:

ncdump -v lat "file://`pwd`/generated/#mode=nczarr,zarr"

ncdump gives me:

netcdf \#mode\=nczarr\,zarr {
dimensions:
	time = 1 ;
	lat = 600 ;
	lon = 1440 ;
variables:
	float Rainf_tavg(time, lat, lon) ;
		Rainf_tavg:cell_methods = "time: mean" ;
		Rainf_tavg:long_name = "Rain precipitation rate" ;
		Rainf_tavg:missing_value = -9999. ;
		Rainf_tavg:standard_name = "rainfall_flux" ;
		Rainf_tavg:units = "kg m-2 s-1" ;
		Rainf_tavg:vmax = 0.00333840004168451 ;
		Rainf_tavg:vmin = 0. ;
	float lat(lat) ;
		lat:axis = "Y" ;
		lat:long_name = "latitude" ;
		lat:standard_name = "latitude" ;
		lat:units = "degrees_north" ;
	float lon(lon) ;
		lon:axis = "X" ;
		lon:long_name = "longitude" ;
		lon:standard_name = "longitude" ;
		lon:units = "degrees_east" ;
	double time(time) ;
		time:axis = "T" ;
		time:bounds = "time_bnds" ;
		time:calendar = "standard" ;
		time:long_name = "time" ;
		time:standard_name = "time" ;
		time:units = "minutes since 2000-01-01T03:00:00" ;

// global attributes:
		:CDI = "Climate Data Interface version 1.9.8 (https://mpimet.mpg.de/cdi)" ;
		:CDO = "Climate Data Operators version 1.9.8 (https://mpimet.mpg.de/cdo)" ;
		:Conventions = "CF-1.6" ;
		:DODS_EXTRA.Unlimited_Dimension = "time" ;
		:DX = 0.25 ;
		:DY = 0.25 ;
		:MAP_PROJECTION = "EQUIDISTANT CYLINDRICAL" ;
		:SOUTH_WEST_CORNER_LAT = -59.875 ;
		:SOUTH_WEST_CORNER_LON = -179.875 ;
		:acc_definision_ = "past 3-hour accumulation" ;
		:comment = "website: https://ldas.gsfc.nasa.gov/gldas, [https://lis.gsfc.nasa.gov/"](https://lis.gsfc.nasa.gov/%22) ;
		:conventions = "CF-1.6" ;
		:history = "created on date: 2019-11-04T10:06:10.461\n2022-07-06 15:11:10 GMT Hyrax-1.16.3 [https://hydro1.gesdisc.eosdis.nasa.gov/opendap/GLDAS/GLDAS_NOAH025_3H.2.1/2000/001/GLDAS_NOAH025_3H.A20000101.0300.021.nc4.nc4?time[0:1:0],lon[0:1:1439],lat[0:1:599],Rainf_tavg[0:1:0][0:1:599][0:1:1439]"](https://hydro1.gesdisc.eosdis.nasa.gov/opendap/GLDAS/GLDAS_NOAH025_3H.2.1/2000/001/GLDAS_NOAH025_3H.A20000101.0300.021.nc4.nc4?time[0:1:0],lon[0:1:1439],lat[0:1:599],Rainf_tavg[0:1:0][0:1:599][0:1:1439]%22) ;
		:inst_definision_ = "instantaneous" ;
		:institution = "NASA GSFC" ;
		:missing_value = -9999. ;
		:references = "Rodell_etal_BAMS_2004, Kumar_etal_EMS_2006, Peters-Lidard_etal_ISSE_2007" ;
		:source = "Noah_v3.6 forced with GDAS-AGRMET-GPCPv13rA1" ;
		:tavg_definision_ = "past 3-hour average" ;
		:title = "GLDAS2.1 LIS land surface model output" ;
data:

 lat = 1.892592e-36, 3.363116e-42, 3.363116e-42, 1.027152e-42, 2.802597e-44, 
    1.821688e-44, 9.187894e-41, 1.301912e+10, 0, 1.040324e-41, 4.706426e-38, 
    2.879904e-39, 2.32714e-29, 6.779446e-31, 2.734375, 5.36916e+08, 
    -3.011738, 1.592435e-37, 1.849173e+28, 7.11524e+25, 2.731982e+23, 
    1.048661e+21, 4.023993e+18, 1.543614e+16, 5.919361e+13, 2.269132e+11, 
    8.695368e+08, 3330835, 12754.07, 48.81667, 0.1867685, 0.000714246, 
    2.730185e-06, 1.043104e-08, 3.98332e-11, 1.520318e-13, 5.799419e-16, 
    2.210975e-18, 8.424034e-21, 3.207596e-23, 1.220529e-25, 4.640977e-28, 
    1.763385e-30, 6.694873e-33, 2.539658e-35, 9.625514e-38, -4.236914e+37, 
    -6.286951e+32, -9.282427e+27, -1.368977e+23, -2.016553e+18, 
    -2.966634e+13, -4.358293e+08, -6393.224, -0.09363129, -1.368862e-06, 
    -1.997411e-11, -2.908485e-16, -4.225408e-21, -6.123076e-26, 
    -8.848101e-31, -1.274586e-35, 3.303676e+35, 7.206622e+25, 1.564891e+16, 
    3380374, 0.00072578, 1.547172e-13, 3.270122e-23, 6.840453e-33, 
    -1.59729e+32, -7.589868e+12, -3.516092e-07, -1.581693e-26, 4.694368e+24, 
    1.022974e-14, -3.22165e+10, 1.902141e-19, -2.000046, -1.05047e-09, 
    1.769917e-24, 8.805741e+14, -2.041029e-31, -4.678544e-12, -1.031469e+08, 
    -2.213423e+27, 2.463612e-35, 1.179822e-25, 5.624584e-16, 2.655094e-06, 
    12431.56, 5.780842e+13, 2.672488e+23, 1.229273e+33, -7.610976e-37, 
    -5.297439e-32, -3.67447e-27, -2.540968e-22, -1.752325e-17, -1.20547e-12, 
    -8.27416e-08, -0.005667652, -387.498, -2.644775e+07, -1.802269e+12, 
    -1.226348e+17, -8.333297e+21, -5.655496e+26, -3.833648e+31, 
    -2.595822e+36, 2.369355e-38, 6.207163e-36, 1.637371e-33, 4.315412e-31, 
    1.136424e-28, 2.990341e-26, 7.862878e-24, 2.066044e-21, 5.425129e-19, 
    1.423664e-16, 3.733745e-14, 9.786652e-12, 2.563819e-09, 6.712972e-07, 
    0.0001756818, 0.04595492, 12.01544, 3140.204, 820340.4, 2.142179e+08, 
    5.591773e+10, 1.45909e+13, 3.805914e+15, 9.92399e+17, 2.586839e+20, 
    6.74083e+22, 1.755994e+25, 4.57302e+27, 1.190578e+30, 3.098784e+32, 
    8.063204e+34, 2.097541e+37, -1.180068e-38, -1.917611e-37, -3.115383e-36, 
    -5.060139e-35, -8.217065e-34, -1.334065e-32, -2.16544e-31, -3.514201e-30, 
    -5.701917e-29, -9.24978e-28, -1.500239e-26, -2.432821e-25, -3.944415e-24, 
    -6.394106e-23, -1.036344e-21, -1.679409e-20, -2.721068e-19, 
    -4.408132e-18, -7.140087e-17, -1.156346e-15, -1.872445e-14, 
    -3.031579e-13, -4.907593e-12, -7.943455e-11, -1.285562e-09, 
    -2.080273e-08, 5.115416e-39, -39.75, 4.658682e-20, 3.355623e+07, 
    1.607556e-40, 7.018674e-38, -0.1259727, 0.7431449, 0.7470588, 3.003921, 
    4.82226e-40, 3.547074e-38, 39.82422, 2.778307e+10, 48.5647, 9.24857e-44, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0 ;
}

[WardF]

Thanks; I'm working on the Fortran release right now but we will take a look!

[DennisHeimbigner]

Since I was not in a position to use python xarray directly, I tried the following instead. Note that I was using the current netcdf-c master, not 4.8.1

1. I used nccopy to create a zarr format file from the original .nc4.
2. I then used ncdump to dump the lat variable values and compared them to a dump of the lat values from the original .nc4. They appear to be the same.

So, I am attaching my zarr file to this message; you might see what python does with it. It is in file directory zarr format.
gldas.tar.gz

[christine-e-smit]

This is the zarr store I generated with xarray:
generated.tar.gz

[christine-e-smit]

Interestingly, when I open and run ncdump on your zarr store, however, I see the same issue:

 % ncdump -v lat "file://`pwd`/gldas.zarr/#mode=nczarr,zarr"
netcdf \#mode\=nczarr\,zarr {
dimensions:
	time = 1 ;
	lat = 600 ;
	lon = 1440 ;
variables:
	float Rainf_tavg(time, lat, lon) ;
		Rainf_tavg:standard_name = "rainfall_flux" ;
		Rainf_tavg:long_name = "Rain precipitation rate" ;
		Rainf_tavg:units = "kg m-2 s-1" ;
		Rainf_tavg:_FillValue = -9999s ;
		Rainf_tavg:missing_value = -9999s ;
		Rainf_tavg:cell_methods = "time: mean" ;
		Rainf_tavg:vmin = 0b ;
		Rainf_tavg:vmax = 0.0033384 ;
	float lat(lat) ;
		lat:standard_name = "latitude" ;
		lat:long_name = "latitude" ;
		lat:units = "degrees_north" ;
		lat:axis = "Y" ;
	float lon(lon) ;
		lon:standard_name = "longitude" ;
		lon:long_name = "longitude" ;
		lon:units = "degrees_east" ;
		lon:axis = "X" ;
	double time(time) ;
		time:standard_name = "time" ;
		time:long_name = "time" ;
		time:bounds = "time_bnds" ;
		time:units = "minutes since 2000-01-01 03:00:00" ;
		time:calendar = "standard" ;
		time:axis = "T" ;

// global attributes:
		:CDI = "Climate Data Interface version 1.9.8 (https://mpimet.mpg.de/cdi)" ;
		:Conventions = "CF-1.6" ;
		:history = "created on date: 2019-11-04T10:06:10.461\n2022-07-06 21:10:36 GMT Hyrax-1.16.3 https://hydro1.gesdisc.eosdis.nasa.gov/opendap/GLDAS/GLDAS_NOAH025_3H.2.1/2000/001/GLDAS_NOAH025_3H.A20000101.0300.021.nc4.nc4?time%5B0:1:0%5D,lon%5B0:1:1439%5D,lat%5B0:1:599%5D,Rainf_tavg%5B0:1:0%5D%5B0:1:599%5D%5B0:1:1439%5D" ;
		:source = "Noah_v3.6 forced with GDAS-AGRMET-GPCPv13rA1" ;
		:institution = "NASA GSFC" ;
		:missing_value = -9999s ;
		:tavg_definision_ = "past 3-hour average" ;
		:acc_definision_ = "past 3-hour accumulation" ;
		:inst_definision_ = "instantaneous" ;
		:title = "GLDAS2.1 LIS land surface model output" ;
		:references = "Rodell_etal_BAMS_2004, Kumar_etal_EMS_2006, Peters-Lidard_etal_ISSE_2007" ;
		:conventions = "CF-1.6" ;
		:comment = "website: https://ldas.gsfc.nasa.gov/gldas, https://lis.gsfc.nasa.gov/" ;
		:MAP_PROJECTION = "EQUIDISTANT CYLINDRICAL" ;
		:SOUTH_WEST_CORNER_LAT = -59.875 ;
		:SOUTH_WEST_CORNER_LON = -179.875 ;
		:DX = 0.25 ;
		:DY = 0.25 ;
		:CDO = "Climate Data Operators version 1.9.8 (https://mpimet.mpg.de/cdo)" ;
		:DODS_EXTRA.Unlimited_Dimension = "time" ;
data:

 lat = -2.56614e+12, 1.693248e+23, -3.48399e+19, 4.913924e+14, 4.476589e+08, 
    -1.508719e-12, -8.53409e-11, -4.633241e-18, 1.981248e+27, 3.540027e+20, 
    -1.389651e-23, -2.866324e-35, -1.43468e-33, -1.190421e-37, 4.878834e-34, 
    -8.140002e+32, 6.609769e+36, -3.709039e+18, -1.014418e-20, -103321.6, 
    -3.143283e-24, -3.27907e-29, 4.61339e-24, 1.174335e-23, 2.887234e-09, 
    6.614554e+23, -38.31552, -3.491201e-29, -8.810143e-20, 1.636443e-29, 
    1.078932e-24, 3.932984e+33, 2.790601e-07, 9.565878e+23, 50408.32, 
    1.062154e+36, -2.723053e-18, 2.437168e+08, -1.699794e-19, -3.065276e-19, 
    -1.483206e-17, 6.089642e+07, -1.929483e-38, 7.450214e-11, 4.822934e-06, 
    6.214098e-31, 7.698601e-07, -4368725, -4.365676e+10, -5.567265e+22, 
    2511562, -1.479679e-26, 2.189595e-14, 5.929955e+11, 1.257928e-19, 
    2.482676e-17, 4.204231e+08, 3.173937e+30, 3.310721e+38, -63762.2, 
    -2.343545e-32, 2.390931e-23, 408.9399, 1.655698e+36, -1.090063e-05, 
    -5.588571e-11, 1.931861e+17, 0.0004408574, 1.632898e+31, -7.394108e+23, 
    -742.1118, -1.774552e-29, 7.461581e-25, 1.13714e+14, 3.223119e+08, 
    -2.289423e+26, 6.453977e+13, 2.12556e-12, 4.896316e+15, 1.071071e-07, 
    4.833929e+23, -1.276742e+09, 289.2994, 1.296792e+21, 7.801811e+09, 
    1.085499e-17, 299812.2, 3.817805e-23, -1748.479, -3809620, 1.641553e-26, 
    3.00377e+34, 4.742595e+34, 9.433684e+30, -9.255541e+22, -434.0548, 
    -6.703581e-35, 2.812479e-30, -4.783806e-37, -2.360466e-16, 0.04597591, 
    6.356505e+19, 9.518348, 4.17818e-22, -1.643167e-24, 0.003336334, 
    3.124911e-15, 3.478393e+12, 7.680207e-17, 2.362477e+15, 2.742291e-32, 
    2.30679e-06, 6.271799e-22, -0.000220317, -9.430286e+30, 3.262875e-16, 
    -3.176065e-19, 693330, 84.81709, 86890.09, 3.799276e-15, 1197202, 
    -3.438033e-17, 1.491627e-31, 2.259057e+22, -5.161287e-16, -4.0269e+07, 
    -1.257379e-23, -5.4747e-10, 5.099318e+22, -885425.6, -2.897339e-25, 
    6.808277e-37, 3.309467e-25, 4.174234e-21, 9.521967e+35, 3.140414e-30, 
    1.293929e-21, 0.0001594756, 5.110542e+29, -1.630333e+22, -6040457, 
    -3.07568e-27, 5.691157e-30, 1.220883e-36, 4.245098e-15, 1.690638e-10, 
    -4.24728e+18, -9.57695e+27, -1.361986e-05, 1.584575e-15, 103154.1, 
    2.634918, -2.701779e-24, 4.267183e-24, 791196.3, 1.796493e-08, 
    -4.623397e+22, 3.640172e-26, -2.036311e-13, 4.03011e+12, -2.654703e+12, 
    -8.041942e-14, -5.443932e-14, -1.036761e+10, -6.254568e-16, -1.48862e-17, 
    14.61257, -1.704033e-19, -1.252717e-15, -2.729831e-18, 1.069042e-16, 
    9.032462e+31, -8.956379e-17, -0.0006886396, -1.512016e+32, -7.146066e+24, 
    -4.783337e+24, 1.165305e-07, 7.802669e-09, 1.622806e+20, -6.407316e+08, 
    -8.58356e-33, 3.205259e-12, -3.149843e-32, -3.294953e-38, -1.221002e-34, 
    102649.6, 0.2225391, -6.588917e+30, -1.646583e-21, -1.250386e-24, 
    -0.1374721, -1.370596e-23, -1.92275e-36, 31781.17, 1.089206e-09, 
    -4.908078e-26, -1.731685e+23, 1.629595e-16, -7.883487e+33, -3.76969e-36, 
    -4.690065e-17, -1.077737e-09, -1.025072e-26, 0.5364485, 9.67961e-28, 
    -1.058745e-35, -52.62312, -7.166411e-12, -1.641529e+34, -6.908097e-26, 
    1.659566e+26, 1.085093e-27, 2.684157e-05, -53.91811, -3.664141e-19, 
    -8.908366e+14, -9.669194e-27, 1.825699e+38, 8.796086e-28, 1.068309e+14, 
    -52.85561, -2.866572e-11, -8.889125e+14, 3629354, -0.002094744, NaNf, 
    2.887902e+19, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 
    0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 ;

[christine-e-smit]

Strangely, when I open your dataset with xarray, the latitude values look fine:

[christine-e-smit]

When I open your zarr store directly with the zarr library, the values also look fine:

[DennisHeimbigner]

Ok, I can think of two reasons for this.

1. netcdf-c is not finding the filter (libz aka deflate) used in the file.
2. the python code is writing information that I did not expect or I am mis-interpreting.

#1 is the most likely cause. If you can send me the zarr file created by your python code, I can see if it is readable by netcdf-c version 4.9.0.

[DennisHeimbigner]

Assuming that filters are the problem, can you tell me where you are storing them?
Perhaps you are setting HDF5_PLUGIN_PATH, and if so, what is its value?

[DennisHeimbigner]

I was able to successfully ncdump the generated file you sent. So
I am pretty sure the problem has to do with filters access. Verion
4.8.1 was still a bit primitive in handling filters, but we should be able
to get it to work.