Refreshing OnBehalfOf token does not work

[HaraldVanWoerkom]

• Package Name: azure.identity.aio
• Package Version: 1.7.1
• Operating System: Windows
• Python Version: 3.8.10

Describe the bug
If the OnBehalfOfCredential from azure.identity.aio is used, the token is not refreshed properly, while OnBehalfOfCredential from azure.identity does refresh the token.

The root cause:
azure.identity.OnBehalfOfCredential uses msal to handle the token, while azure.identity.aio.OnBehalfOfCredential uses an internal AadClient. This AadClient does two things different from msal:

• Msal automatically adds the openid, profile, and offline_access scopes
• Msal includes the client_secret in the refresh request

Without the client_secret, the authority server returns:
AADSTS7000218: The request body must contain the following parameter: 'client_assertion' or 'client_secret'.
I cannot find a reference to this error and the documentation also suggests that it should not be needed for refresh tokens, but the error itself is self-explanatory.
Note that OnBehalfOfCredential swallows this error and continues using the old token. When this token finally expires we get an error that the assertion token has expired (which is correct, but this hides the underlying problem).

To Reproduce
Steps to reproduce the behavior:

1. Create an OnBehalfOfCredential object with client_secret
2. Create an object that uses this async credential, in our case adlfs.AzureBlobFileSystem
3. Every minute, do an operation on the object that requires a token (e.g. exists()
4. After 60-90 minutes, an exception is raised that explains that the assertion token is out-of-date

Expected behavior
The operation should continue to succeed for a long time (refresh tokens are long-running tokens)

Screenshots

Additional context
Note that if the synchronous variant of OnBehalfOfCredential is used and an object that uses it (e.g. BlobServiceClient), the token is refreshed properly.

By adding the client_secret and extending the scopes of the token (I do not know if the latter is actually needed), the library can be made to work (I tried this with some hack code, but I'm not familiar enough with the code to suggest a proper solution).

[YalinLi0312]

@HaraldVanWoerkom Thanks for your feedback, we'll investigate asap.

[mccoyp]

Hi @HaraldVanWoerkom, thank you for opening an issue! I'll try to reproduce this error and get back to you as soon as I can.

[HaraldVanWoerkom]

Is there any progress on this issue?

[xiangyan99]

@HaraldVanWoerkom we are still working on reproducing the issue. May I ask if you were using same registered app for sync and async OboCredentials?

I asked this because sometimes AADSTS7000218 may mean the app is not correctly configured. https://nishantrana.me/2020/12/01/fixed-aadsts7000218-the-request-body-must-contain-the-following-parameter-client_assertion-or-client_secret/.

Thanks.

[HaraldVanWoerkom]

Yes, I am using the same registered app.
Note that the current code of the AadClient used for the async token update just cannot work, since it does not have the client_secret. If you hack the code so it does have the client_secret, it works.

I tried the following hack, which I do not like. But it does get the job done:
azure\identity\aio_credentials\on_behalf_of.py: OnBehalfOfCredential._request_token, line 100:
scopes = [s for s in scopes] + ['openid', 'profile', 'offline_access']
line 105 (after line 100 was added):
kwargs["client_secret"] = self._client_credential

azure\identity_internal\aad_client_base.py: aad_client_base._get_refresh_token_request, line 257
if "client_secret" in kwargs:
data["client_secret"] = kwargs["client_secret"]
kwargs.pop("client_secret")
azure\identity_internal\aad_client.py: aad_client.obtain_refresh_token, line 72
if "client_secret" in kwargs:
kwargs.pop("client_secret")

[xiangyan99]

Thanks for the investigation.

I opened a PR #23551.

If possible, could you try if it works for you?

Thanks.