fix(python, rust): remove imds calls from profile auth and region

[mightyshazam]

Description

The AWS SDK uses EC2 instance metadata in the default provider chain, the profile chain and the region provider

Related Issue(s)

• closes Very slow s3 connection after 0.16.1 #2377

Documentation

[ion-elgreco]

Hey @mightyshazam, I am seeing also a lot of warnings with our latest release regarding this IMDS region:

[2024-04-23T06:13:29Z WARN  aws_config::imds::region] failed to load region from IMDS ...
[2024-04-23T06:13:29Z WARN  aws_config::imds::region] failed to load region from IMDS ...

Would it make sense to have this IMDS region disabled by default? A lot of users including me don't use AWS s3 but other variants

Right, so this PR makes it not the default anymore or you still need to manually pass disable_EC2_metadata everytime? (sorry just woke up : ) )

[mightyshazam]

This PR means you have to pass it. I am tempted to make disabling ec2 metadata default behavior for the sake of backwards compatibility, though.

[ion-elgreco]

@mightyshazam let's go for putting it disabled by default. Then for EC2 instance users they should enable it.

[ion-elgreco]

Thanks @mightyshazam!

[ion-elgreco]

@mightyshazam I checked today, but I am still seeing some warnings. So it seems to be not fully disabled yet?

[2024-04-24T07:55:28Z WARN  aws_config::imds::region] failed to load region from IMDS err=failed to load IMDS session token: dispatch failure: io error: error trying to connect: tcp connect error: No route to host (os error 113): tcp connect error: No route to host (os error 113): No route to host (os error 113) (FailedToLoadToken(FailedToLoadToken { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Io, source: hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 113, kind: HostUnreachable, message: "No route to host" })), connection: Unknown } }) }))
[2024-04-24T07:55:29Z WARN  aws_config::imds::region] failed to load region from IMDS err=failed to load IMDS session token: dispatch failure: timeout: error trying to connect: HTTP connect timeout occurred after 1s: HTTP connect timeout occurred after 1s: timed out (FailedToLoadToken(FailedToLoadToken { source: DispatchFailure(DispatchFailure { source: ConnectorError { kind: Timeout, source: hyper::Error(Connect, HttpTimeoutError { kind: "HTTP connect", duration: 1s }), connection: Unknown } }) }))

[mightyshazam]

@ion-elgreco I don't see this when running with default settings. I also don't see it when setting AWS_EC2_METADATA_DISABLED to false. However, I was able to reproduce it by setting the environment variable AWS_EC2_METADATA_DISABLED set to false. Can you check your environment variables?

[ion-elgreco]

Only when setting the env variable AWS_EC2_METADATA_DISABLED=true will stop trying to use that region provider. If I don't set any environment variables, the warnings will pop up again during a connection