unable to authenticate OpenDAP

[alexander-buerger-met-no]

We are using Ubuntu 16.04, 18.04 and 20.04 and noticed that OpenDAP authentication got worse and worse:

• in 16.04 (xenial), both netrc and user:pass in URL work
• in 18.04 (bionic), netrc stopped working, but user:pass in URL still works
• in 20.04 (focal), none of these works, we are not aware of any way to authenticate with username and password
• we have the impression that authentication does not work with the conda-forge libnetcdf package either

The problem can be reproduced like this:

docker run -it ubuntu:bionic # or ubuntu:focal
# in the container:
apt update && apt install -y netcdf-bin
ncdump -ct 'https://user:pass@server/path' # insert the actual username and password and url here

With ubuntu:bionic, this lists the contents. With ubuntu:focal, the exact same ncdump command prints Authorization failure.

NetCDF versions are 4.6.0 on bionic and 4.7.3 on focal. libcurl version is 7.58.0 on bionic and focal.

[alexander-buerger-met-no]

Update: there were mistakes in the above message.

• in 20.04 (focal), only netrc seems to work
• with libnetcdf from conda-forge, it is the same as for Ubuntu 20.04, i.e. only netrc works

The example docker commands are only for user:pass, of course. The setup for netrc is similar, but a little more complicated.

The NetCDF version on conda-forge is 4.7.4.

[WardF]

@DennisHeimbigner I'll defer to you and your expertise on this before trying to dive in. Any ideas?

[DennisHeimbigner]

what is the contents of your .dodsrc and .netrc?
[don't forget to X out the passwords]

[alexander-buerger-met-no]

I attached an example running ncdump in a container. After inserting username and password (in netrc) and url path (in Dockerfile.focal), the example can be run with bash run.sh focal or bash run.sh bionic.

example.zip

[DennisHeimbigner]

This may take some time. Our password protected example on thredds-test seems
to have vanished and I need to get it re-established before I can debug this.

[alexander-buerger-met-no]

Ok, thanks.

In the example from the zip file, password from netrc works in focal while password in url fails in focal. For bionic, it is the other way around: password from netrc does not work, but password in url works.

[DennisHeimbigner]

Unfortunately, I am not in a position to run ubuntu 18 or 20.
So for the moment, I will not be able to work on this problem.
[to clarify, I need to re-partition my disk to make room for more VMs]

[DennisHeimbigner]

BTW is there anyway you can back down to curl 7.57 to see if that also fails?

[alexander-buerger-met-no]

Ok. No hurry. If you do not need to repartition for other reasons, maybe somebody else can run the docker containers to confirm the problem?

As the curl package comes from Ubuntu, I cannot easily downgrade to 7.57.

[DennisHeimbigner]

I still can't even get a working server using basic user:pwd. So I have no idea if the original fault can be duplicated.

[DennisHeimbigner]

Sigh! I finally got everything into place and working.
Sadly under ubuntu 18.04, every combination I can try appears to work.
So let me check:

1. .netrc is in your home directory
2. .dodsrc is in your local directory or in your home directory
3. no other occurrences of .netrc or .dodsrc or .daprc exist in either home or local directories

[DennisHeimbigner]

Oops, I realized that I was using netcdf 4.8; let me try 4.6.0