Skip to content

Lambda Recursion Detection will not work with Lambda Managed Instances #3354

@bmoffatt

Description

@bmoffatt

Acknowledgements

Describe the bug

Lambda recursion detection, added to aws-sdk-go-v2 by #2105 in 2023, took a dependency on the environment variable _X_AMZN_TRACE_ID. For customers using Lambda Managed Instances, for which support was added in github.com/aws/aws-lambda-go@v1.51.0, the aws-sdk-go-v2 recursion detection will no longer work, as the environment variable will not be present. The environment variable cannot be used to pass the trace id, as Lambda Managed Instances environments handle concurrent requests within the same process.

Good news, the context.Context passed to handlers by github.com/aws/aws-lambda-go, includes the trace id. This is how the integration was done with the (recently deprecated) https://github.com/aws/aws-xray-sdk-go.

traceID, ok := ctx.Value("x-amzn-trace-id").(string)

Here's where it is set today: https://github.com/aws/aws-lambda-go/blob/v1.53.0/lambda/invoke_loop.go#L72

And where it was set in 2017: https://github.com/aws/aws-lambda-go/blob/v1.0.0/lambda/function.go#L53

That _X_AMZN_TRACE_ID is also set, is a more 'recent' (2020) addition, done to resolve a documentation inconsistency aws/aws-lambda-go#269

Regression Issue

  • Select this option if this issue appears to be a regression.

Expected Behavior

X-Amzn-Trace-Id to be added to request headers when the sdk is used with Lambda Managed Instances

Current Behavior

Header will not be present

Reproduction Steps

N/A

Possible Solution

I suggest switching aws-sdk-go-v2 to looking for the context value, and falling back to the environment variable for back-compat with customers who're passing context.Background() rather than the lambda provided context.Context.

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

N/A

Compiler and Version used

N/A

Operating System and version

N/A

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugThis issue is a bug.p2This is a standard priority issuequeuedThis issues is on the AWS team's backlog

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions