nc-config command-line options

[wkliao]

Questions about two command-line options of utility program nc-config.
At first, I assume one of the purposes of nc-config is to display the compile and link options used when building the NetCDF libraries. For example, options --cflags and --libs may include the environment variables CFLAGS and LIBS set at configure time. If this is the case, then --cflags and --libs should not include the installation path of NetCDF. The installation paths can be obtained from the already provided option --includedir and --libdir.

Below are my configure command and output from nc-config when using the master branch.

% ./configure --prefix=$HOME/NetCDF \
          --enable-netcdf-4 \
          CC=mpicc \
          CFLAGS="-g -Wall" \
          CPPFLAGS="-I/homes/wkliao/HDF5/include" \
          LDFLAGS="-L/homes/wkliao/HDF5/lib"

% ./nc-config --all
  --cflags    -> -I/homes/wkliao/NetCDF/include -I/homes/wkliao/HDF5/include
  --libs      -> -L/homes/wkliao/NetCDF/lib -lnetcdf

I expected the followings.

% ./nc-config --all
  --cflags        -> -g -Wall
  --cppflags      -> -I/homes/wkliao/HDF5/include 
  --ldflags       -> -L/homes/wkliao/HDF5/lib 
  --libs          ->
...
  --prefix        -> /homes/wkliao/NetCDF
  --includedir    -> /homes/wkliao/NetCDF/include
  --libdir        -> /homes/wkliao/NetCDF/lib

Please let me know if my above understanding of nc-config is correct.

[WardF]

You are correct in the purpose of nc-config. I don't know what the standard/best practices usage is for building nc-config files like this (I will review when I have time, this predates me at Unidata), but I suspect that these paths are included because the do not do any harm and are simply erring on the side of being overly verbose. Removing these paths from nc-config --libs, etc, at this point would require modifying any extant workflows so that then call --includedir and/or --libdir.

Is this an observation you've had, or is there an issue/problem being caused by this that you've seen?

[edhartnett]

My understanding is that the output of nc-config --libs should be able to be used in user programs in the compile line; similarly for --cppflags. So they include the netCDF install directories so that a user program can do something like:

gcc nc-config --cppflags -o my_program my_program.c nc-config --libs

And the compile will work. (github is taking away my backticks around the nc-config commands in the command line above...)

[wkliao]

PnetCDF has a new feature soon available in 1.11.0 release that allows PnetCDF to be built on top of NetCDF to allow PnetCDF APIs to access HDF-based files. nc-config is convenient for figuring out the include and lib paths and any extra libraries used to build NetCDF. That way, PnetCDF users are only required to provide the NetCDF installation path. All other components can then be automatically retrieved through nc-config.

NCO appears to use this approach in its configure.ac as well.

Hi, @edhartnett That is also my understanding, too.

[wkliao]

At the end of NetCDF configure command, the contents of file libnetcdf.settings are dumped on screen. Below shows the one from my build. It appears nc-config can use the same settings.

# Compiling Options
-----------------
C Compiler:             /homes/wkliao/MPICH/bin/mpicc
CFLAGS:                 -g -Wall
CPPFLAGS:               -I/homes/wkliao/HDF5/include
LDFLAGS:                -L/homes/wkliao/HDF5/lib
AM_CFLAGS:
AM_CPPFLAGS:
AM_LDFLAGS:
Shared Library:         yes
Static Library:         yes
Extra libraries:        -lhdf5_hl -lhdf5 -lm -lcurl
...

[wkliao]

Hi, @edhartnett
To show a back-tick character in Markdown language, just add the escape character \ before it. For example, I revised your compile command into

> gcc \`nc-config --cppflags\` -o my_program my_program.c \`nc-config --libs\`

and it now becomes:

gcc `nc-config --cppflags` -o my_program my_program.c `nc-config --libs`

[wkliao]

On second thought, I do not think option --cppflags and --libs are intended to be used by NetCDF user programs. For example, a NetCDF program only needs the NetCDF include path in its compile command line, but not the HDF5 include path. The right way to use nc-config in compile command line is:

gcc -o my_program my_program.c -I`nc-config --includedir` -L`nc-config --libdir` -lnetcdf

Options --cppflags and --libs can be useful for a 3rd party library, such as PnetCDF and NCO, to check the version of HDF5 library and any "Extra libraries" (e.g. -lhdf5_hl -lhdf5 -lm -lcurl) used to build HDF5 or NetCDF.

[tenomoto]

With the shared library of libnetcdf.dylib on Mac, adding Extra libraries cause overlinking. A user can safely use just -lnetcdf as a result of nc-config --libs to build a program. Perhaps rewrite CMakeLists.txt line 1849 to avoid adding ${NC_LIBS} if shared?

[edhartnett]

It's not clear to me that other build systems should use nc-config to get around the normal methods of specifying a library location.

Without nc-config, the user has to install netCDF in /usr/local/ or else specify the include directory in CPPFLAGS and the library directory in LDFLAGS.

When using nc-config, it must be in the path, which is just as much trouble as specifying CPPFLAGS and LDFLAGS. So what is the advantage of using nc-config?

We have a challenge here in that shared and static builds are different. For shared builds, only -lnetcdf need be provided, and it will find the HDF5, zlib, and potentially szlib. But for static builds, all must be specified on the link line.

Finally, we cannot just change behavior for static vs. shared builds, because shared builds also build the static library. So it's not static vs shared, it's static vs. shared + static.

[Dave-Allured]

Starting with netcdf-c new release 4.6.3, pkg-config solves this problem of static vs. shared linking, and overlinking. A pkg-config bug was recently fixed; see #1324. Users and other build systems should use pkg-config, not nc-config, to build their link commands. For shared builds:

$ pkg-config --libs netcdf

And for static builds, add the --static option:

$ pkg-config --libs --static netcdf

See pkg-config docs. This guide to pkg-config is helpful.

[edhartnett]

@Dave-Allured that is awesome.

So should nc-config just be left as it is, this ticket closed, and users directed to use pkg-config instead?

[Dave-Allured]

Good question. I am not sure. Nc-config is a component of netcdf-c, whereas pkg-config is an external standard utility. I don't know the history of nc-config vs. pkg-config within netCDF. My guess is that nc-config should be fixed in a similar way for shared vs. static. I forgot to ask about nc-config when we were discussing the other in #1324.

Who is familiar with the purpose of nc-config, and would it be appropriate to add a --static option?

[WardF]

There's no harm in leaving nc-config, although we could add behavior similar to pkg-config. I'm inclined to close this ticket out since we have solved the issue with pkg-config, and I'll open a new one reflecting this updated issue! Thanks all.