{data}[gompi/2022.10] netCDF-Fortran v4.6.0, netCDF v4.9.0, HDF5 v1.12.2, Szip v2.1.1 by boegel · Pull Request #16834 · easybuilders/easybuild-easyconfigs

boegel · 2022-12-08T11:01:02Z

(created using eb --new-pr)
requires:

{devel,lib}[GCCcore/12.2.0] Doxygen v1.9.5, libiconv v1.17 #16825

…2.2, Szip v2.1.1

boegel · 2022-12-08T11:02:20Z

@boegelbot please test @ generoso

boegelbot · 2022-12-08T11:05:12Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=16834 EB_ARGS= EB_CONTAINER= /opt/software/slurm/bin/sbatch --job-name test_PR_16834 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 9718

Test results coming soon (I hope)...

Details

- notification for comment with ID 1342538205 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-12-08T11:39:38Z

Test report by @boegelbot
FAILED
Build succeeded for 2 out of 4 (4 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/454fa3603b82904c59a5a44082f9c4b7 for a full test report.

…asyconfigs into 20221208120057_new_pr_netCDF-Fortran460

branfosj · 2022-12-08T19:12:17Z

Test report by @branfosj
SUCCESS
Build succeeded for 7 out of 7 (4 easyconfigs in total)
bear-pg0211u15a.bear.cluster - Linux Ubuntu 20.04.2 LTS (Focal Fossa), x86_64, Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz (cascadelake), Python 3.8.5
See https://gist.github.com/cce047459e4433f1f1cc1791cfe1102f for a full test report.

branfosj · 2022-12-08T19:16:39Z

Test report by @branfosj
SUCCESS
Build succeeded for 7 out of 7 (4 easyconfigs in total)
bear-pg0105u36b.bear.cluster - Linux RHEL 8.5, x86_64, Intel(R) Xeon(R) Platinum 8360Y CPU @ 2.40GHz (icelake), Python 3.6.8
See https://gist.github.com/a9a985bf072e63c2ab85027b9189dcd6 for a full test report.

branfosj · 2022-12-08T19:16:58Z

@boegelbot please test @ jsc-zen2

boegelbot · 2022-12-08T19:17:08Z

@branfosj: Request for testing this PR well received on jsczen2l1.int.jsc-zen2.easybuild-test.cluster

PR test command 'EB_PR=16834 EB_ARGS= /opt/software/slurm/bin/sbatch --mem-per-cpu=4000M --job-name test_PR_16834 --ntasks=8 ~/boegelbot/eb_from_pr_upload_jsc-zen2.sh' executed!

exit code: 0
output:

Submitted batch job 1853

Test results coming soon (I hope)...

Details

- notification for comment with ID 1343226691 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-12-08T19:48:45Z

Test report by @boegelbot
FAILED
Build succeeded for 2 out of 4 (4 easyconfigs in total)
jsczen2g1.int.jsc-zen2.easybuild-test.cluster - Linux Rocky Linux 8.5, x86_64, AMD EPYC 7742 64-Core Processor (zen2), Python 3.6.8
See https://gist.github.com/ec1025e47b4a8dd5776c8eb9f7e10ed5 for a full test report.

boegel · 2022-12-09T13:02:23Z

Same test fails on both generoso and jsc-zen2, doesn't look like timeout is too strict:

161/218 Test #161: nc_test4_run_par_test .................***Timeout 1500.15 sec
...
The following tests FAILED:
	161 - nc_test4_run_par_test (Timeout)

In the past it seems like this was a sign of a bug in Open MPI, see Unidata/netcdf-c#1500.
But here it succeeds on most systems...

branfosj · 2022-12-09T13:23:42Z

@boegelbot please test @ generoso
CORE_CNT=16

boegelbot · 2022-12-09T13:25:11Z

@branfosj: Request for testing this PR well received on login1

PR test command 'EB_PR=16834 EB_ARGS= EB_CONTAINER= /opt/software/slurm/bin/sbatch --job-name test_PR_16834 --ntasks="16" ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 9728

Test results coming soon (I hope)...

Details

- notification for comment with ID 1344306828 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

branfosj · 2022-12-09T13:25:21Z

We disabled one of the parts of nc_test4_run_par_test in #16050, but we are not even getting that far. This part of the tests runs really slowly for some - seeing if more cores on generoso helps.

boegelbot · 2022-12-09T13:57:33Z

Test report by @boegelbot
FAILED
Build succeeded for 2 out of 4 (4 easyconfigs in total)
cnx1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz (haswell), Python 3.6.8
See https://gist.github.com/ea8e72aa79c7aa8e0b67f109422feda1 for a full test report.

boegel · 2022-12-12T19:31:48Z

@branfosj How do you think we should proceed here? Extend the timeout?

branfosj · 2022-12-13T11:36:45Z

@branfosj How do you think we should proceed here? Extend the timeout?

We should try with a longer timeout - to see how long it takes for these tests to complete. I expect that we'll just keep increasing it for the tests in here - I supsect that the functionality these tests hit are a bad fit for certain types of filesystems.

…22.10

boegel · 2022-12-20T19:54:48Z

@boegelbot please test @ generoso

boegelbot · 2022-12-20T19:55:09Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=16834 EB_ARGS= EB_CONTAINER= /opt/software/slurm/bin/sbatch --job-name test_PR_16834 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 9852

Test results coming soon (I hope)...

Details

- notification for comment with ID 1360078495 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegel · 2022-12-20T22:14:40Z

Test report by @boegel
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
node3104.skitty.os - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/9a852e73965ea63f7b1099fff2ef353a for a full test report.

…on some types of filesystems

boegel · 2022-12-21T08:27:41Z

I have cancelled job 9852 that was running on generoso with relatex timeout (ARGS='--timeout 100000' in pretestopts), since it was already running for 12.5h.

Instead, I've added a new patch in d721297 which disable one more test, because it's running excessively long on some types of filesystems.

boegel · 2022-12-21T08:27:48Z

@boegelbot please test @ generoso

boegelbot · 2022-12-21T08:30:11Z

@boegel: Request for testing this PR well received on login1

PR test command 'EB_PR=16834 EB_ARGS= EB_CONTAINER= /opt/software/slurm/bin/sbatch --job-name test_PR_16834 --ntasks=4 ~/boegelbot/eb_from_pr_upload_generoso.sh' executed!

exit code: 0
output:

Submitted batch job 9856

Test results coming soon (I hope)...

Details

- notification for comment with ID 1361001272 processed

Message to humans: this is just bookkeeping information for me,
it is of no use to you (unless you think I have a bug, which I don't).

boegelbot · 2022-12-21T09:04:40Z

Test report by @boegelbot
FAILED
Build succeeded for 2 out of 4 (4 easyconfigs in total)
cns1 - Linux Rocky Linux 8.5, x86_64, Intel(R) Xeon(R) CPU E5-2667 v3 @ 3.20GHz (haswell), Python 3.6.8
See https://gist.github.com/7aafcc29d42601ec98c6e59b75ee6d75 for a full test report.

boegel · 2022-12-21T09:13:00Z

Test report by @boegel
SUCCESS
Build succeeded for 4 out of 4 (4 easyconfigs in total)
node3104.skitty.os - Linux RHEL 8.6, x86_64, Intel(R) Xeon(R) Gold 6140 CPU @ 2.30GHz (skylake_avx512), Python 3.6.8
See https://gist.github.com/846087f611eb1ad12fbc4784aebf2512 for a full test report.

boegel · 2022-12-21T15:51:04Z

+ and machine-independent data formats that support the creation, access, and sharing of array-oriented
+ scientific data."""
+
+toolchain = {'name': 'gompi', 'version': '2022.10'}


Note: version should be updated to 2022b once #16961 is merged

SebastianAchilles · 2023-01-04T09:45:23Z

I am also seeing the timeout on my system. In my case I am running the test report in a container and the filesystem is mounted into the container, e.g. similar to:

docker run -it -v /data/easybuild/:/home/easybuild/.local/easybuild/ ghcr.io/easybuilders/rockylinux-9.0:latest

/data is a 4TB SATA HDD with an ext4 filesystem. Using --buildpath=/dev/shm/$USER/ doesn't make a difference.

I tried to nail down the tests that are talking so long. To get rid of the timeout I had to disable in addition to the already disabled tests the @MPIEXEC@ -n 4 ./tst_parallel_zlib and @MPIEXEC@ -n 4 ./tst_parallel_compress tests, e.g. I added the following to the patch netCDF-4.9.0_skip-timeout-tests.patch:

@@ -50,14 +50,14 @@
 # Only run these tests if HDF5 supports parallel filters (v1.10.2 and
 # later).
 if test "@HAS_PAR_FILTERS@" = "yes"; then
-    echo
-    echo "Parallel I/O test with zlib."
-    @MPIEXEC@ -n 4 ./tst_parallel_zlib
+#     echo
+#     echo "Parallel I/O test with zlib."
+#     @MPIEXEC@ -n 4 ./tst_parallel_zlib
 
     echo
     echo "Parallel I/O more tests with zlib and szip (if present in HDF5)."
     @MPIEXEC@ -n 1 ./tst_parallel_compress
-    @MPIEXEC@ -n 4 ./tst_parallel_compress
+#     @MPIEXEC@ -n 4 ./tst_parallel_compress
 fi
 
 echo

Skipping these tests also works on jsc-zen2, which is using a NFS filesystem.

Micket · 2023-01-04T10:43:09Z

I would have thought the netCDF-4.9.0_skip-nasa-test.patch patch was enough. That seemed to be enough for netCDF-4.9.0-gompi-2022a.eb and netCDF-4.9.0-iimpi-2022a.eb

It did fix things for the person who originated this issue a while back #15959

If it's not enough, then we should apply this new patch to those easyconfigs as well.

SebastianAchilles · 2023-01-04T11:04:34Z

Yes, in the 2022a toolchain skipping only the Parallel Performance Test for NASA for netCDF-4.9.0-gompi-2022a.eb worked on generoso, jsc-zen2, and my container that I am using for test reports. For the version for 2022b more tests seem to take longer. The exact reason why the tests take longer is not clear for me.

Micket · 2023-01-04T12:00:02Z

So the difference stems from the compiler and MPI change, or the version bump of libxml2 and cURL.
I would suspect MPI if anything.

Maybe it's not so clear cut that these failures should just be ignored?

I'll try on cephfs, to see if that also has problems without these patches (it never had issues with any of the tests before).

boegel · 2023-02-28T15:47:10Z

closing in favor of @jfgrimm's #17107 (which just got merged)...

{data}[gompi/2022.10] netCDF-Fortran v4.6.0, netCDF v4.9.0, HDF5 v1.1…

81f5473

…2.2, Szip v2.1.1

boegel added the update label Dec 8, 2022

boegel added this to the next release (4.7.0) milestone Dec 8, 2022

boegel mentioned this pull request Dec 8, 2022

{geo}[foss/2022b] WRF v4.4.1, netCDF-Fortran 4.6.0, tcsh v6.24.05, time v1.9 #16835

Merged

Merge branch 'develop' of https://github.com/easybuilders/easybuild-e…

d55093f

…asyconfigs into 20221208120057_new_pr_netCDF-Fortran460

easybuilders deleted a comment from boegelbot Dec 8, 2022

increase timeout for tests in netCDF v4.9.0 easyconfig using gompi/20…

efdb237

…22.10

disable another test for netCDF 4.9.0 because it runs excessive long …

d721297

…on some types of filesystems

boegel force-pushed the 20221208120057_new_pr_netCDF-Fortran460 branch from 18fa679 to d721297 Compare December 21, 2022 08:27

boegel commented Dec 21, 2022

View reviewed changes

boegel modified the milestones: next release (4.7.0), release after 4.7.0 Jan 3, 2023

branfosj mentioned this pull request Jan 13, 2023

{data}[gompi/2022b,iimpi/2022b] HDF5 v1.14.0, netCDF v4.9.0 #17107

Merged

1 task

boegel closed this Feb 28, 2023

boegel deleted the 20221208120057_new_pr_netCDF-Fortran460 branch February 28, 2023 15:47

boegel modified the milestones: next release (4.7.1?), 4.x Feb 28, 2023

Conversation

boegel commented Dec 8, 2022

Uh oh!

boegel commented Dec 8, 2022

Uh oh!

boegelbot commented Dec 8, 2022

Uh oh!

boegelbot commented Dec 8, 2022

Uh oh!

branfosj commented Dec 8, 2022

Uh oh!

branfosj commented Dec 8, 2022

Uh oh!

branfosj commented Dec 8, 2022

Uh oh!

boegelbot commented Dec 8, 2022

Uh oh!

boegelbot commented Dec 8, 2022

Uh oh!

boegel commented Dec 9, 2022

Uh oh!

branfosj commented Dec 9, 2022

Uh oh!

boegelbot commented Dec 9, 2022

Uh oh!

branfosj commented Dec 9, 2022 • edited by boegel Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegelbot commented Dec 9, 2022

Uh oh!

boegel commented Dec 12, 2022

Uh oh!

branfosj commented Dec 13, 2022

Uh oh!

boegel commented Dec 20, 2022

Uh oh!

boegelbot commented Dec 20, 2022

Uh oh!

boegel commented Dec 20, 2022

Uh oh!

boegel commented Dec 21, 2022

Uh oh!

boegel commented Dec 21, 2022

Uh oh!

boegelbot commented Dec 21, 2022

Uh oh!

boegelbot commented Dec 21, 2022

Uh oh!

boegel commented Dec 21, 2022

Uh oh!

boegel Dec 21, 2022

Choose a reason for hiding this comment

Uh oh!

SebastianAchilles commented Jan 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Micket commented Jan 4, 2023

Uh oh!

SebastianAchilles commented Jan 4, 2023

Uh oh!

Micket commented Jan 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

boegel commented Feb 28, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

branfosj commented Dec 9, 2022 •

edited by boegel

Loading

SebastianAchilles commented Jan 4, 2023 •

edited

Loading

Micket commented Jan 4, 2023 •

edited

Loading