Merge branch 'stratus.dmh' of https://github.com/DennisHeimbigner/netcdf-c into gh2152.wif

WardF · WardF · commit 4d13f8ef7e94 · 2022-01-25T11:20:20.000-07:00
diff --git a/RELEASE_NOTES.md b/RELEASE_NOTES.md
@@ -11,7 +11,8 @@ This file contains a high-level description of this package's evolution. Release
 * [Bug Fix] Make sure that netcdf.h accurately defines the flags in the open/create mode flags. See [Github #2183](https://github.com/Unidata/netcdf-c/pull/2183).
 * [Enhancement] Improve support for msys2+mingw platform. See [Github #2171](https://github.com/Unidata/netcdf-c/pull/2171).
 * [Bug Fix] Clean up the various inter-test dependencies in ncdump for CMake. See [Github #2168](https://github.com/Unidata/netcdf-c/pull/2168).
-* [Enhancement] Added options to suppress the new behavior from [Github #2135](https://github.com/Unidata/netcdf-c/pull/2135).  The options for `cmake` and `configure` are, respectively `-DENABLE_LIBXML2` and `--(enable/disable)-libxml2`. Both of these options defaul to 'on/enabled'.  When disabled, the bundled `ezxml` XML interpreter is used regardless of whether `libxml2` is present on the system. 
+* [Bug Fix] Fix use of non-aws appliances. See [Github #2152](https://github.com/Unidata/netcdf-c/pull/2152).
+* [Enhancement] Added options to suppress the new behavior from [Github #2135](https://github.com/Unidata/netcdf-c/pull/2135).  The options for `cmake` and `configure` are, respectively `-DENABLE_LIBXML2` and `--(enable/disable)-libxml2`. Both of these options defaul to 'on/enabled'.  When disabled, the bundled `ezxml` XML interpreter is used regardless of whether `libxml2` is present on the system.
 * [Enhancement] Support optional use of libxml2, otherwise default to ezxml. See [Github #2135](https://github.com/Unidata/netcdf-c/pull/2135) -- H/T to [Egbert Eich](https://github.com/e4t).
 * [Bug Fix] Fix several os related errors. See [Github #2138](https://github.com/Unidata/netcdf-c/pull/2138).
 * [Enhancement] Support byte-range reading of netcdf-3 files stored in private buckets in S3. See [Github #2134](https://github.com/Unidata/netcdf-c/pull/2134)
@@ -29,24 +30,24 @@ This file contains a high-level description of this package's evolution. Release
 * [Enhancement] Convert to a new representation of the NCZarr meta-data extensions: version 2. Read-only backward compatibility is provided. See [Github #2032](https://github.com/Unidata/netcdf-c/pull/2032).
 * [Bug Fix] Fix dimension_separator bug in libnczarr. See [Github #2035](https://github.com/Unidata/netcdf-c/pull/2035).
 * [Bug Fix] Fix bugs in libdap4. See [Github #2005](https://github.com/Unidata/netcdf-c/pull/2005).
-* [Bug Fix] Store NCZarr fillvalue as a singleton instead of a 1-element array. See [Github #2017](https://github.com/Unidata/netcdf-c/pull/2017).  
+* [Bug Fix] Store NCZarr fillvalue as a singleton instead of a 1-element array. See [Github #2017](https://github.com/Unidata/netcdf-c/pull/2017).
 * [Bug Fixes] The netcdf-c library was incorrectly determining the scope of dimension; similar to the type scope problem. See [Github #2012](https://github.com/Unidata/netcdf-c/pull/2012) for more information.
-* [Bug Fix] Re-enable DAP2 authorization testing. See [Github #2011](https://github.com/Unidata/netcdf-c/issues/2011).  
-* [Bug Fix] Fix bug with windows version of mkstemp that causes failure to create more than 26 temp files. See [Github #1998](https://github.com/Unidata/netcdf-c/pull/1998).  
-* [Bug Fix] Fix ncdump bug when printing VLENs with basetype char. See [Github #1986](https://github.com/Unidata/netcdf-c/issues/1986).  
+* [Bug Fix] Re-enable DAP2 authorization testing. See [Github #2011](https://github.com/Unidata/netcdf-c/issues/2011).
+* [Bug Fix] Fix bug with windows version of mkstemp that causes failure to create more than 26 temp files. See [Github #1998](https://github.com/Unidata/netcdf-c/pull/1998).
+* [Bug Fix] Fix ncdump bug when printing VLENs with basetype char. See [Github #1986](https://github.com/Unidata/netcdf-c/issues/1986).
 * [Bug Fixes] The netcdf-c library was incorrectly determining the scope of types referred to by nc_inq_type_equal. See [Github #1959](https://github.com/Unidata/netcdf-c/pull/1959) for more information.
 * [Bug Fix] Fix bug in use of XGetopt when building under Mingw. See [Github #2009](https://github.com/Unidata/netcdf-c/issues/2009).
 * [Enhancement] Improve the error reporting when attempting to use a filter for which no implementation can be found in HDF5_PLUGIN_PATH. See [Github #2000](https://github.com/Unidata/netcdf-c/pull/2000) for more information.
 * [Bug Fix] Fix `make distcheck` issue in `nczarr_test/` directory. See [Github #2007](https://github.com/Unidata/netcdf-c/issues/2007).
 * [Bug Fix] Fix bug in NCclosedir in dpathmgr.c. See [Github #2003](https://github.com/Unidata/netcdf-c/issues/2003).
-* [Bug Fix] Fix bug in ncdump that assumes that there is a relationship between the total number of dimensions and the max dimension id. See [Github #2004](https://github.com/Unidata/netcdf-c/issues/2004).  
-* [Bug Fix] Fix bug in JSON processing of strings with embedded quotes. See [Github #1993](https://github.com/Unidata/netcdf-c/issues/1993).  
+* [Bug Fix] Fix bug in ncdump that assumes that there is a relationship between the total number of dimensions and the max dimension id. See [Github #2004](https://github.com/Unidata/netcdf-c/issues/2004).
+* [Bug Fix] Fix bug in JSON processing of strings with embedded quotes. See [Github #1993](https://github.com/Unidata/netcdf-c/issues/1993).
 * [Enhancement] Add support for the new "dimension_separator" enhancement to Zarr v2. See [Github #1990](https://github.com/Unidata/netcdf-c/pull/1990) for more information.
-* [Bug Fix] Fix hack for handling failure of shell programs to properly handle escape characters. See [Github #1989](https://github.com/Unidata/netcdf-c/issues/1989).  
-* [Bug Fix] Allow some primitive type names to be used as identifiers depending on the file format. See [Github #1984](https://github.com/Unidata/netcdf-c/issues/1984).  
+* [Bug Fix] Fix hack for handling failure of shell programs to properly handle escape characters. See [Github #1989](https://github.com/Unidata/netcdf-c/issues/1989).
+* [Bug Fix] Allow some primitive type names to be used as identifiers depending on the file format. See [Github #1984](https://github.com/Unidata/netcdf-c/issues/1984).
 * [Enhancement] Add support for reading/writing pure Zarr storage format that supports the XArray _ARRAY_DIMENSIONS attribute. See [Github #1952](https://github.com/Unidata/netcdf-c/pull/1952) for more information.
 * [Update] Updated version of bzip2 used in filter testing/functionality, in support of [Github #1969](https://github.com/Unidata/netcdf-c/issues/1969).
-* [Bug Fix] Corrected HDF5 version detection logic as described in [Github #1962](https://github.com/Unidata/netcdf-c/issues/1962).  
+* [Bug Fix] Corrected HDF5 version detection logic as described in [Github #1962](https://github.com/Unidata/netcdf-c/issues/1962).
 
 ## 4.8.0 - March 30, 2021
 
diff --git a/include/ncs3sdk.h b/include/ncs3sdk.h
@@ -10,8 +10,8 @@
 extern "C" {
 #endif
 
-EXTERNL void NC_s3sdkinitialize(void);
-EXTERNL void NC_s3sdkfinalize(void);
+EXTERNL int NC_s3sdkinitialize(void);
+EXTERNL int NC_s3sdkfinalize(void);
 EXTERNL void* NC_s3sdkcreateclient(NCS3INFO* context);
 EXTERNL int NC_s3sdkbucketexists(void* s3client, const char* bucket, int* existsp, char** errmsgp);
 EXTERNL int NC_s3sdkbucketcreate(void* s3client, const char* region, const char* bucket, char** errmsgp);
diff --git a/libdispatch/ds3util.c b/libdispatch/ds3util.c
@@ -119,11 +119,12 @@ NC_s3urlrebuild(NCURI* url, NCURI** newurlp, char** bucketp, char** outregionp)
 	if((stat = NC_getdefaults3region(url,&region0))) goto done;
 	region = strdup(region0);
     }
-    /* Construct the revised host */
-    ncbytescat(buf,"s3.");
-    ncbytescat(buf,region);
-    ncbytescat(buf,AWSHOST);
-    host = ncbytesextract(buf);
+    if(host == NULL) { /* Construct the revised host */
+        ncbytescat(buf,"s3.");
+        ncbytescat(buf,region);
+        ncbytescat(buf,AWSHOST);
+        host = ncbytesextract(buf);
+    }
 
     /* Construct the revised path */
     ncbytesclear(buf);
diff --git a/libdispatch/ncs3sdk.cpp b/libdispatch/ncs3sdk.cpp
@@ -65,7 +65,7 @@ dumps3info(NCS3INFO* info)
     return text;
 }
 
-EXTERNL void
+EXTERNL int
 NC_s3sdkinitialize(void)
 {
     if(!ncs3_initialized) {
@@ -75,9 +75,10 @@ NC_s3sdkinitialize(void)
 	Aws::InitAPI(ncs3options);
 	NCUNTRACE(NC_NOERR);
     }
+    return 1;
 }
 
-EXTERNL void
+EXTERNL int
 NC_s3sdkfinalize(void)
 {
     if(!ncs3_finalized) {
@@ -87,6 +88,7 @@ NC_s3sdkfinalize(void)
         Aws::ShutdownAPI(ncs3options);
         NCUNTRACE(NC_NOERR);
     }
+    return 1;
 }
 
 static char*
diff --git a/liblib/nc_initialize.c b/liblib/nc_initialize.c
@@ -47,8 +47,8 @@ extern int NC_HDF4_finalize(void);
 #endif
 
 #ifdef ENABLE_S3_SDK
-extern int NC_s3sdkinitialize(void);
-extern int NC_s3sdkfinalize(void);
+EXTERNL int NC_s3sdkinitialize(void);
+EXTERNL int NC_s3sdkfinalize(void);
 #endif
 
 #ifdef _MSC_VER
diff --git a/ncdap_test/Makefile.am b/ncdap_test/Makefile.am
@@ -10,6 +10,7 @@ include $(top_srcdir)/lib_flags.am
 #SH_LOG_DRIVER = $(SHELL) $(top_srcdir)/test-driver-verbose
 #LOG_DRIVER = $(SHELL) $(top_srcdir)/test-driver-verbose
 #TEST_LOG_DRIVER = $(SHELL) $(top_srcdir)/test-driver-verbose
+#TESTS_ENVIRONMENT += export SETX=1;
 
 # Note which tests depend on other tests. Necessary for make -j check.
 TEST_EXTENSIONS = .sh
diff --git a/nczarr_test/run_interop.sh b/nczarr_test/run_interop.sh
@@ -69,19 +69,22 @@ case "$zext" in
 	testcasezip ref_quotes zarr metaonly
 	;;
     s3)
-	# Read a test case created by netcdf-java zarr.
-	# Move into position
-        rm -f ${execdir}/ref_zarr_test_data.cdl
-        if gunzip -c < ${srcdir}/ref_zarr_test_data.cdl.gz > ${execdir}/ref_zarr_test_data.cdl ; then
-            testcases3 zarr_test_data.zarr ref_zarr_test_data xarray
+	# Test file does not exist on stratus
+	if test "x$NCZARR_S3_TEST_HOST" = "xs3.us-east-1.amazonaws.com" ; then
+  	    # Read a test case created by netcdf-java zarr.
+	    # Move into position
+            rm -f ${execdir}/ref_zarr_test_data.cdl
+            if gunzip -c < ${srcdir}/ref_zarr_test_data.cdl.gz > ${execdir}/ref_zarr_test_data.cdl ; then
+                testcases3 zarr_test_data.zarr ref_zarr_test_data xarray
+            fi
         fi
 	;;
     *) echo "unimplemented kind: $1" ; exit 1;;
 esac
 }
 
-#testallcases file
-#if test "x$FEATURE_NCZARR_ZIP" = xyes ; then testallcases zip; fi
+testallcases file
+if test "x$FEATURE_NCZARR_ZIP" = xyes ; then testallcases zip; fi
 if test "x$FEATURE_S3TESTS" = xyes ; then testallcases s3; fi
 exit
 # Cleanup

Original file line number	Diff line number	Diff line change
`@@ -65,7 +65,7 @@ dumps3info(NCS3INFO* info)`
`65`	`65`	`return text;`
`66`	`66`	`}`
`67`	`67`
`68`		`-EXTERNL void`
	`68`	`+EXTERNL int`
`69`	`69`	`NC_s3sdkinitialize(void)`
`70`	`70`	`{`
`71`	`71`	`if(!ncs3_initialized) {`
`@@ -75,9 +75,10 @@ NC_s3sdkinitialize(void)`
`75`	`75`	`Aws::InitAPI(ncs3options);`
`76`	`76`	`NCUNTRACE(NC_NOERR);`
`77`	`77`	`}`
	`78`	`+ return 1;`
`78`	`79`	`}`
`79`	`80`
`80`		`-EXTERNL void`
	`81`	`+EXTERNL int`
`81`	`82`	`NC_s3sdkfinalize(void)`
`82`	`83`	`{`
`83`	`84`	`if(!ncs3_finalized) {`
`@@ -87,6 +88,7 @@ NC_s3sdkfinalize(void)`
`87`	`88`	`Aws::ShutdownAPI(ncs3options);`
`88`	`89`	`NCUNTRACE(NC_NOERR);`
`89`	`90`	`}`
	`91`	`+ return 1;`
`90`	`92`	`}`
`91`	`93`
`92`	`94`	`static char*`