nc_create("file://foo.zarr#mode=nczarr,noxarray", 0, &ncid) consumes all memory

[opoplawski]

We are seeing this with netcdf 4.9.0 and current git main head on Fedora Rawhide. The following is a simple reproducer:

#include <netcdf.h>

int main(int argc, char **argv) {
   int ncid;
   nc_create("file://foo.zarr#mode=nczarr,noxarray", 0, &ncid);
}

This is derived from a test in python-xarray - an issue was initially raised there: pydata/xarray#6854

[opoplawski]

It seems to get stuck in an infinite allocation/copy loop. backtrace:

#0  _int_malloc (av=av@entry=0x7ffff7bf6c80 <main_arena>, bytes=bytes@entry=7) at malloc.c:4368
#1  0x00007ffff7abe392 in __GI___libc_malloc (bytes=bytes@entry=7) at malloc.c:3297
#2  0x00007ffff7ac451e in __GI___strdup (s=s@entry=0x45bab0 "nczarr") at strdup.c:42
#3  0x00007ffff7c4edfc in processinferences (fraglenv=0x45bba0) at ../libdispatch/../../libdispatch/dinfermodel.c:526
#4  NC_infermodel (path=0x45b680 "file://foo.zarr#mode=nczarr,noxarray", omodep=<optimized out>, iscreate=1, useparallel=0, params=0x0, model=0x7fffffffd800, newpathp=0x7fffffffd7f0)
    at ../libdispatch/../../libdispatch/dinfermodel.c:852
#5  0x00007ffff7c42c66 in NC_create (path0=<optimized out>, cmode=<optimized out>, initialsz=0, basepe=basepe@entry=0, chunksizehintp=0x0, useparallel=useparallel@entry=0, parameters=0x0, ncidp=0x7fffffffd88c)
    at ../libdispatch/../../libdispatch/dfile.c:1866
#6  0x00007ffff7c42e69 in nc__create (path=<optimized out>, cmode=<optimized out>, initialsz=<optimized out>, chunksizehintp=<optimized out>, ncidp=<optimized out>) at ../libdispatch/../../libdispatch/dfile.c:469

[opoplawski]

Pointer to the python-xarray test: https://github.com/pydata/xarray/blob/main/xarray/tests/test_backends.py#L5733

[DennisHeimbigner]

ok, I have a fix.But it has revealed an odd HDF5 error that I need to track down.

[DennisHeimbigner]

Fixed by PR #2574