Incomplete library exports on Windows

[letmaik]

Environment Information

• What platform are you using? (please provide specific distribution/version in summary)
  • Linux
  • Windows
  • OSX
  • Other
  • NA
• 32 and/or 64 bit?
  • 32-bit
  • 64-bit
• What build system are you using?
  • autotools (configure)
  • cmake

Summary of Issue

The pre-built netcdf-c libraries at http://www.unidata.ucar.edu/software/netcdf/docs/winbin.html are missing certain DLL symbols like nc_create_par_fortran, nc_open_par_fortran, nc_var_par_access. This can be verified with the following command: dumpbin /exports libnetcdf.dll.

I also built netcdf-c 4.5.0 in an MSYS/mingw64 environment using cmake and faced the same problem.

Not having those functions exported means that it is impossible to link netcdf-fortran against it as it requires those.

I couldn't exactly figure out yet what the reason is but I noticed that there are two files which contain overlapping function declarations with different export keywords: include/netcdf_par.h (not included without parallel option) and include/netcdf_f.h (always included). Maybe that gives a clue?

[WardF]

Thank you for reporting; I will get this fixed!

[edhartnett]

Those functions are only used for parallel I/O builds of netCDF. Did you intend to use parallel IO on Windows? That is not typical, as far as I know.

So it seems like the windows DLL should not include those functions.

[letmaik]

Thanks for looking at it.

No I didn't want to use parallel IO but it seems that netcdf-fortran has a hard dependency on those functions, see https://github.com/Unidata/netcdf-fortran/blob/78a9839366cf8cdc929da450ebba9fad941c3341/fortran/nf_nc4.f90#L61. This file is included whenever USE_NETCDF4 is enabled, see https://github.com/Unidata/netcdf-fortran/blob/78a9839366cf8cdc929da450ebba9fad941c3341/fortran/Makefile.am#L58.

From what I can see, netcdf_h.h is included unconditionally in libdispatch in dvar.c which is always built.

netcdf-c/libdispatch/dvar.c

Lines 8 to 9 in 327c194

	#include "ncdispatch.h"
	#include "netcdf_f.h"

Shouldn't that then mean that those functions are always included irrelevant of the parallel IO build flag?

[WardF]

We currently don't officially support Fortran on Windows, which is why this hasn't received as much attention. But I should be able to address that.

[WardF]

I assume you are using mingw64/fortran equivalent and not Visual Studio or some such via the MSYS shell?

[letmaik]

I use the mingw64 shell from the MSYS2 package. Note that the person from the linked issue used the Intel Fortran compiler on Windows and had the same problem. Make sure to build netcdf-fortran as shared library, otherwise you may not notice the undefined symbols.

[WardF]

Part of the problem is a lack of access to free fortran compilers compatible with visual studio; I understand that PGI has released a community edition of their compilers, so when I get a chance I will try to get it working with that.

[letmaik]

OK great, I didn't know that about the PGI compiler. On their feature comparison page it says that the free version doesn't have Visual Studio integration.
Feel free to contact me via email if I can be of any help with reproducing it or setting up mingw64. I'm currently a bit blocked by this issue and would really like to get this solved. Once fixed, netcdf-fortran could be added to the msys/mingw package repository which would make it a lot easier to build scientific software on Windows that uses netcdf-fortran.

[dmey]

Hi @WardF, I have encountered the same issue with the installation of netcdf-fortran as reported above.
Could you please let me know if this issue is currently being actively looked at?
If not, is this something that you may be able to shed some light on? -- Perhaps we can try to address the issue collaboratively? Thank you.

[WardF]

Thanks, I have mingw64 set up and will gladly take a look at this as soon as I can; it is definitely being looked at, but there are a number of issues and pull requests currently being reviewed as well. My assumption is that the functions just need to be exported as other functions have needed to be on (on Windows). @letmaik just for the sake of completion, can you share how you are building netcdf-c on Windows with cmake using MSYS2/mingw64? I work in MSYS2 already, so getting it set up won't be a problem.

[dmey]

Thanks @WardF, if it is of any help, these are the installation steps I followed:

1. Open the MSYS2 MSYS command prompt and install dependencies using the pacman package manager

pacman -S mingw-w64-x86_64-toolchain
pacman -S mingw-w64-x86_64-netcdf

2. Open the the MSYS2 MinGW 64-bit command prompt to configure netcdf-fortran with CMake

cd /tmp
wget ftp://ftp.unidata.ucar.edu/pub/netcdf/netcdf-fortran-4.4.4.tar.gz
tar xvzf netcdf-fortran-4.4.4.tar.gz
cd netcdf-fortran-4.4.4
mkdir build && cd build
cmake -G "MSYS Makefiles" -DCMAKE_BUILD_TYPE=Release -DENABLE_TESTS=OFF ..

which produces the following output:

-- The Fortran compiler identification is GNU 7.2.0
-- The C compiler identification is GNU 7.2.0
-- Check for working Fortran compiler: C:/msys64/mingw64/bin/gfortran.exe
-- Check for working Fortran compiler: C:/msys64/mingw64/bin/gfortran.exe  -- works
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Checking whether C:/msys64/mingw64/bin/gfortran.exe supports Fortran 90
-- Checking whether C:/msys64/mingw64/bin/gfortran.exe supports Fortran 90 -- yes
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe
-- Check for working C compiler: C:/msys64/mingw64/bin/gcc.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Could NOT find Doxygen (missing: DOXYGEN_EXECUTABLE)
-- Looking for nc_def_opaque in netcdf
-- Looking for nc_def_opaque in netcdf - not found
-- Looking for nccreate in netcdf
-- Looking for nccreate in netcdf - not found
-- Looking for nc_set_log_level in netcdf
-- Looking for nc_set_log_level in netcdf - not found
-- Looking for oc_open in netcdf
-- Looking for oc_open in netcdf - not found
-- Looking for nc_use_parallel_enabled in netcdf
-- Looking for nc_use_parallel_enabled in netcdf - not found


-- CMake Summary:

--      Build Type:                     Release
--      Building Shared Libraries:      ON
--      Building netCDF-4:
--      Building DAP Support:
--      Logging Support:                OFF
--      Using Parallel IO:              OFF
--      Linking against:                netcdf;

-- Configuring done
-- Generating done
-- Build files have been written to: C:/msys64/tmp/netcdf-fortran-4.4.4/build

3. Install

make VERBOSE=1

which fails with the following output:

--- tail ---
[ 41%] Linking Fortran shared library libnetcdff.dll
cd /C/msys64/tmp/netcdf-fortran-4.4.4/build/fortran && /C/msys64/mingw64/bin/cmake.exe -E remove -f CMakeFiles/netcdff.dir/objects.a
cd /C/msys64/tmp/netcdf-fortran-4.4.4/build/fortran && /C/msys64/mingw64/bin/ar.exe cr CMakeFiles/netcdff.dir/objects.a "CMakeFiles/netcdff.dir/typeSizes.f90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nc_data.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nc_interfaces.f90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nf_data.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf_nf_interfaces.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf_f03.f90.obj" "CMakeFiles/netcdff.dir/nf_attio.F90.obj" "CMakeFiles/netcdff.dir/nf_control.F90.obj" "CMakeFiles/netcdff.dir/nf_dim.f90.obj" "CMakeFiles/netcdff.dir/nf_misc.f90.obj" "CMakeFiles/netcdff.dir/nf_genatt.f90.obj" "CMakeFiles/netcdff.dir/nf_geninq.f90.obj" "CMakeFiles/netcdff.dir/nf_genvar.f90.obj" "CMakeFiles/netcdff.dir/nf_vario.F90.obj" "CMakeFiles/netcdff.dir/nf_var1io.F90.obj" "CMakeFiles/netcdff.dir/nf_varaio.F90.obj" "CMakeFiles/netcdff.dir/nf_varmio.F90.obj" "CMakeFiles/netcdff.dir/nf_varsio.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf4_nc_interfaces.f90.obj" "CMakeFiles/netcdff.dir/module_netcdf4_nf_interfaces.F90.obj" "CMakeFiles/netcdff.dir/module_netcdf4_f03.f90.obj" "CMakeFiles/netcdff.dir/nf_lib.c.obj" "CMakeFiles/netcdff.dir/nf_nc4.f90.obj" "CMakeFiles/netcdff.dir/netcdf4.f90.obj" "CMakeFiles/netcdff.dir/nf_v2compat.c.obj" "CMakeFiles/netcdff.dir/module_netcdf_fortv2_c_interfaces.f90.obj" "CMakeFiles/netcdff.dir/nf_fortv2.f90.obj"
cd /C/msys64/tmp/netcdf-fortran-4.4.4/build/fortran && /C/msys64/mingw64/bin/gfortran.exe  -O2 -g -DNDEBUG  -shared -o libnetcdff.dll -Wl,--out-implib,libnetcdff.dll.a -Wl,--major-image-version,6,--minor-image-version,1 -Wl,--whole-archive CMakeFiles/netcdff.dir/objects.a -Wl,--no-whole-archive /C/msys64/mingw64/lib/libnetcdf.dll.a /mingw64/lib/libhdf5_hl.dll.a /mingw64/lib/libhdf5.dll.a /mingw64/lib/libz.dll.a -lgcc_eh -lgcc_eh
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_create_par_':
C:/msys64/tmp/netcdf-fortran-4.4.4/fortran/nf_nc4.f90:61: undefined reference to `nc_create_par_fortran'
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_open_par_':
C:/msys64/tmp/netcdf-fortran-4.4.4/fortran/nf_nc4.f90:94: undefined reference to `nc_open_par_fortran'
CMakeFiles/netcdff.dir/objects.a(nf_nc4.f90.obj): In function `nf_var_par_access_':
C:/msys64/tmp/netcdf-fortran-4.4.4/fortran/nf_nc4.f90:121: undefined reference to `nc_var_par_access'
collect2.exe: error: ld returned 1 exit status
make[2]: *** [fortran/CMakeFiles/netcdff.dir/build.make:779: fortran/libnetcdff.dll] Error 1
make[2]: Leaving directory '/tmp/netcdf-fortran-4.4.4/build'
make[1]: *** [CMakeFiles/Makefile2:983: fortran/CMakeFiles/netcdff.dir/all] Error 2
make[1]: Leaving directory '/tmp/netcdf-fortran-4.4.4/build'
make: *** [Makefile:141: all] Error 2