Slower `nc_open` with version 4.9.3

[trexfeathers]

Slower nc_open with version 4.9.3

Hello, thanks for your work on this wonderful package, and for the heroic efforts getting 4.9.3 onto conda-forge 😊

The nc_open() function looks to be significantly slower in version 4.9.3 versus 4.9.2.

Recorded times on my machine

• Small dummy file (see CDL below): 400 microseconds slower; 2100 versus 1700
• A1B_north_america.nc: 5 milliseconds slower; 15 versus 10

dummy.nc CDL

Expand for CDL

netcdf simple_example {
  dimensions:
    x = 10;
    y = 10;
  variables:
    float data(x, y);
  data:
    data = 1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
           11, 12, 13, 14, 15, 16, 17, 18, 19, 20,
           21, 22, 23, 24, 25, 26, 27, 28, 29, 30,
           31, 32, 33, 34, 35, 36, 37, 38, 39, 40,
           41, 42, 43, 44, 45, 46, 47, 48, 49, 50,
           51, 52, 53, 54, 55, 56, 57, 58, 59, 60,
           61, 62, 63, 64, 65, 66, 67, 68, 69, 70,
           71, 72, 73, 74, 75, 76, 77, 78, 79, 80,
           81, 82, 83, 84, 85, 86, 87, 88, 89, 90,
           91, 92, 93, 94, 95, 96, 97, 98, 99, 100 ;
}

Why these small slowdowns still matter

SciTools/iris provides parallel computation+streaming of NetCDF files in distributed compute setups. This has seen successful operational use for several years (ping @bouweandela, @valeriupredoi). For this use-case, the small slowdowns seen here scale to multiple seconds for parallel computation of 1000 chunks or more.

Explanation

The distributed compute nodes must not simultaneously open a single NetCDF file for writing, since that causes concurrency errors at the HDF5 level. Such errors can be prevented with appropriate use of locking, and ensuring that each parallel task opens and closes the NetCDF file independently - i.e. the file must not be left open after a parallel task completes. Thus, 1000 chunks = 1000 calls to nc_open() = 1000x slowdown.

I have tried to mitigate this slowdown by having tasks 'share' an already open file handle (with careful locking), but I am not aware¹ of a way to achieve this 'sharing' between distributed compute nodes.

Info for reproducing

NetCDF config

Expand for `nc-config --all` output

This netCDF 4.9.3 has been built with the following features: 

  --cc               -> x86_64-conda-linux-gnu-cc
  --cflags           -> -I/home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493/include
  --libs             -> -L/home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493/lib -lnetcdf
  --static           -> -ljpeg -ldf -lmfhdf -lHDF5::HDF5 -lhdf5::hdf5_hl -lm -lz -lzip -lblosc -lzstd -lbz2 -lsz -lCURL::libcurl -lxml2


  --has-dap          -> yes
  --has-dap2         -> yes
  --has-dap4         -> yes
  --has-nc2          -> yes
  --has-nc4          -> yes
  --has-hdf5         -> yes
  --has-hdf4         -> yes
  --has-logging      -> no
  --has-pnetcdf      -> no
  --has-szlib        -> yes
  --has-cdf5         -> yes
  --has-parallel4    -> no
  --has-parallel     -> no
  --has-nczarr       -> yes
  --has-zstd         -> yes
  --has-benchmarks   -> yes
  --has-multifilters -> yes
  --has-stdfilters   -> deflate szip blosc zstd bz2
  --has-quantize     -> yes

  --prefix            -> /home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493
  --includedir        -> /home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493/include
  --libdir            -> /home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493/lib
  --plugindir         -> /home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493/hdf5/lib/plugin
  --plugin-searchpath -> /home/users/martin.yeo/.conda/envs/tmp_libnetcdf_493/hdf5/lib/plugin:/usr/local/hdf5/lib/plugin
  --version           -> netCDF 4.9.3
  --build-system      -> cmake

Environment

OS: RHEL9

gcc info: gcc (conda-forge gcc 15.2.0-7) 15.2.0

Expand for Conda environment spec

# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                 conda_forge    conda-forge
_openmp_mutex             4.5                       2_gnu    conda-forge
attr                      2.5.2                h39aace5_0    conda-forge
binutils_impl_linux-64    2.44                 hdf8817f_2    conda-forge
blosc                     1.21.6               he440d0b_1    conda-forge
bzip2                     1.0.8                hda65f42_8    conda-forge
c-ares                    1.34.5               hb9d3cd8_0    conda-forge
ca-certificates           2025.10.5            hbd8a1cb_0    conda-forge
conda-gcc-specs           15.2.0               h56430cd_7    conda-forge
gcc                       15.2.0               hc115cf6_7    conda-forge
gcc_impl_linux-64         15.2.0               hcacfade_7    conda-forge
hdf4                      4.2.15               h2a13503_7    conda-forge
hdf5                      1.14.6          nompi_h6e4c0c1_103    conda-forge
icu                       75.1                 he02047a_0    conda-forge
kernel-headers_linux-64   5.14.0               he073ed8_2    conda-forge
keyutils                  1.6.3                hb9d3cd8_0    conda-forge
krb5                      1.21.3               h659f571_0    conda-forge
ld_impl_linux-64          2.44                 ha97dd6f_2    conda-forge
libaec                    1.1.4                h3f801dc_0    conda-forge
libcurl                   8.14.1               h332b0f4_0    conda-forge
libedit                   3.1.20250104    pl5321h7949ede_0    conda-forge
libev                     4.33                 hd590300_2    conda-forge
libgcc                    15.2.0               h767d61c_7    conda-forge
libgcc-devel_linux-64     15.2.0             h73f6952_107    conda-forge
libgcc-ng                 15.2.0               h69a702a_7    conda-forge
libgfortran               15.2.0               h69a702a_7    conda-forge
libgfortran5              15.2.0               hcd61629_7    conda-forge
libgomp                   15.2.0               h767d61c_7    conda-forge
libiconv                  1.18                 h3b78370_2    conda-forge
libjpeg-turbo             3.1.0                hb9d3cd8_0    conda-forge
liblzma                   5.8.1                hb9d3cd8_2    conda-forge
libnetcdf                 4.9.3           nompi_h11f7409_103    conda-forge
libnghttp2                1.67.0               had1ee68_0    conda-forge
libsanitizer              15.2.0               hb13aed2_7    conda-forge
libssh2                   1.11.1               hcf80075_0    conda-forge
libstdcxx                 15.2.0               h8f9b012_7    conda-forge
libstdcxx-ng              15.2.0               h4852527_7    conda-forge
libxml2                   2.15.0               h26afc86_1    conda-forge
libxml2-16                2.15.0               ha9997c6_1    conda-forge
libzip                    1.11.2               h6991a6a_0    conda-forge
libzlib                   1.3.1                hb9d3cd8_2    conda-forge
lz4-c                     1.10.0               h5888daf_1    conda-forge
ncurses                   6.5                  h2d0b736_3    conda-forge
openssl                   3.5.4                h26f9b46_0    conda-forge
snappy                    1.2.2                h03e3b7b_0    conda-forge
sysroot_linux-64          2.34                 h087de78_2    conda-forge
tzdata                    2025b                h78e105d_0    conda-forge
zlib                      1.3.1                hb9d3cd8_2    conda-forge
zstd                      1.5.7                hb8e6e7a_2    conda-forge

Code

(With help from Copilot as my C skills are limited).

#include <netcdf.h>
#include <stdio.h>
#include <time.h>

#define N_RUNS 25

double time_nc_open(const char *filename) {
    struct timespec start, end;
    int ncid, retval;
    clock_gettime(CLOCK_MONOTONIC, &start);
    retval = nc_open(filename, NC_NOWRITE, &ncid);
    if (retval != NC_NOERR) {
        fprintf(stderr, "nc_open error: %s\n", nc_strerror(retval));
        return -1;
    }
    nc_close(ncid);
    clock_gettime(CLOCK_MONOTONIC, &end);
    return (end.tv_sec - start.tv_sec) + (end.tv_nsec - start.tv_nsec) / 1e6;
}

int main(int argc, char **argv) {
    if (argc != 2) {
        printf("Usage: %s <netcdf_file>\n", argv[0]);
        return 1;
    }
    double total = 0;
    int errors = 0;
    for (int i = 0; i < N_RUNS; ++i) {
        double t = time_nc_open(argv[1]);
        if (t < 0) {
            errors++;
            continue;
        }
        total += t;
    }
    printf("Average nc_open time: %.2f milliseconds over %d runs\n", total / (N_RUNS - errors), N_RUNS - errors);
    if (errors > 0) printf("nc_open failed %d times\n", errors);
    return 0;
}

Compiling

gcc -o ncopen_bench ncopen_bench.c -lnetcdf

Footnotes

1. We are working at the Python level. ↩