Skip to content

Feature to update maximum_batching_window_in_seconds in aws lambda event source mapping for SQS event source type#16518

Merged
anGie44 merged 7 commits into
hashicorp:masterfrom
with-joy:f-aws_lambda_event_source_mapping
Dec 2, 2020
Merged

Feature to update maximum_batching_window_in_seconds in aws lambda event source mapping for SQS event source type#16518
anGie44 merged 7 commits into
hashicorp:masterfrom
with-joy:f-aws_lambda_event_source_mapping

Conversation

@with-joy

@with-joy with-joy commented Dec 1, 2020

Copy link
Copy Markdown

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates OR Closes #16503

Release note for CHANGELOG:

resource/aws_lambda_event_source_mapping: Updating maximum_batching_window_in_seconds for SQS event source type

Output from acceptance testing:

 make testacc TESTARGS='-run=TestAccAWSLambdaEventSourceMapping_SQSBatchWindow'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSLambdaEventSourceMapping_SQSBatchWindow -timeout 120m
=== RUN   TestAccAWSLambdaEventSourceMapping_SQSBatchWindow
=== PAUSE TestAccAWSLambdaEventSourceMapping_SQSBatchWindow
=== CONT  TestAccAWSLambdaEventSourceMapping_SQSBatchWindow
--- PASS: TestAccAWSLambdaEventSourceMapping_SQSBatchWindow (85.65s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       87.373s

...

@with-joy with-joy requested a review from a team as a code owner December 1, 2020 22:47
@ghost ghost added size/M Managed by automation to categorize the size of a PR. service/lambda Issues and PRs that pertain to the lambda service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Dec 1, 2020
@github-actions github-actions Bot added the needs-triage Waiting for first response or review from a maintainer. label Dec 1, 2020
@with-joy with-joy changed the title F aws lambda event source mapping Feature to update maximum_batching_window_in_seconds in aws lambda event source mapping for SQS event source type Dec 1, 2020

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @with-joy 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@anGie44 anGie44 added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Dec 2, 2020

@anGie44 anGie44 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @with-joy for this PR! looks great so far 👍 just a couple comments and request to also update the resource's documentation in lambda_event_source_mapping.html.markdown as well as it currently is targeted towards streaming resources e.g.

The maximum amount of time to gather records before invoking the function, in seconds (between 0 and 300). 
Records will continue to buffer (or accumulate in the case of an SQS queue event source) until either `maximum_batching_window_in_seconds` expires or `batch_size` has been met. 
For streaming event sources, defaults to as soon as records are available in the stream. 
If the batch it reads from the stream/queue only has one record in it, Lambda only sends one record to the function.

Output of acceptance tests:

--- PASS: TestAccAWSLambdaEventSourceMapping_sqs_withFunctionName (45.41s)
--- PASS: TestAccAWSLambdaEventSourceMapping_SQSBatchWindow (66.68s)
--- PASS: TestAccAWSLambdaEventSourceMapping_StartingPositionTimestamp (71.71s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqs_basic (82.33s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_disappears (86.47s)
--- PASS: TestAccAWSLambdaEventSourceMapping_KinesisDestinationConfig (94.02s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_removeBatchSize (94.10s)
--- PASS: TestAccAWSLambdaEventSourceMapping_BatchWindow (94.78s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_basic (95.49s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqsDisappears (99.38s)
--- PASS: TestAccAWSLambdaEventSourceMapping_changesInEnabledAreDetected (99.68s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRecordAgeInSeconds (105.76s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRetryAttempts (106.70s)
--- PASS: TestAccAWSLambdaEventSourceMapping_ParallelizationFactor (107.48s)
--- PASS: TestAccAWSLambdaEventSourceMapping_BisectBatch (108.67s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRetryAttemptsZero (109.05s)

Comment thread aws/resource_aws_lambda_event_source_mapping_test.go Outdated
Comment thread aws/resource_aws_lambda_event_source_mapping_test.go Outdated
Comment thread aws/resource_aws_lambda_event_source_mapping.go Outdated
Comment thread aws/resource_aws_lambda_event_source_mapping_test.go
@anGie44 anGie44 self-assigned this Dec 2, 2020
@with-joy

with-joy commented Dec 2, 2020

Copy link
Copy Markdown
Author

Thank you @anGie44 for all the suggestions, was really helpful 🙏🏾
Here's the acceptance test results after the changes:

--- PASS: TestAccAWSLambdaEventSourceMapping_SQSBatchWindow (122.02s)
--- PASS: TestAccAWSLambdaEventSourceMapping_KinesisDestinationConfig (147.40s)
--- PASS: TestAccAWSLambdaEventSourceMapping_KinesisBatchWindow (162.85s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_basic (168.87s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRecordAgeInSeconds (174.72s)
--- PASS: TestAccAWSLambdaEventSourceMapping_BisectBatch (192.91s)
--- PASS: TestAccAWSLambdaEventSourceMapping_StartingPositionTimestamp (200.35s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_disappears (202.28s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqs_withFunctionName (209.64s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqs_basic (210.29s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRetryAttempts (214.77s)
--- PASS: TestAccAWSLambdaEventSourceMapping_ParallelizationFactor (218.39s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRetryAttemptsZero (220.52s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_removeBatchSize (229.09s)
--- PASS: TestAccAWSLambdaEventSourceMapping_changesInEnabledAreDetected (251.29s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqsDisappears (255.15s)
PASS
ok

@with-joy with-joy requested a review from anGie44 December 2, 2020 13:19
@anGie44 anGie44 added this to the v3.20.0 milestone Dec 2, 2020

@anGie44 anGie44 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the updates @with-joy , look great 🚀

Output of acceptance tests:

--- PASS: TestAccAWSLambdaEventSourceMapping_BisectBatch (204.35s)
--- PASS: TestAccAWSLambdaEventSourceMapping_KinesisBatchWindow (208.77s)
--- PASS: TestAccAWSLambdaEventSourceMapping_KinesisDestinationConfig (213.08s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRecordAgeInSeconds (213.65s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRetryAttempts (200.18s)
--- PASS: TestAccAWSLambdaEventSourceMapping_MaximumRetryAttemptsZero (228.37s)
--- PASS: TestAccAWSLambdaEventSourceMapping_ParallelizationFactor (213.56s)
--- PASS: TestAccAWSLambdaEventSourceMapping_SQSBatchWindow (157.08s)
--- PASS: TestAccAWSLambdaEventSourceMapping_StartingPositionTimestamp (135.33s)
--- PASS: TestAccAWSLambdaEventSourceMapping_changesInEnabledAreDetected (165.21s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_basic (206.03s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_disappears (126.40s)
--- PASS: TestAccAWSLambdaEventSourceMapping_kinesis_removeBatchSize (212.73s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqsDisappears (161.15s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqs_basic (197.35s)
--- PASS: TestAccAWSLambdaEventSourceMapping_sqs_withFunctionName (101.37s)

@anGie44 anGie44 merged commit ec8ef3e into hashicorp:master Dec 2, 2020
anGie44 added a commit that referenced this pull request Dec 2, 2020
@with-joy with-joy deleted the f-aws_lambda_event_source_mapping branch December 3, 2020 07:00
@ghost

ghost commented Dec 3, 2020

Copy link
Copy Markdown

This has been released in version 3.20.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

@ghost

ghost commented Jan 2, 2021

Copy link
Copy Markdown

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked as resolved and limited conversation to collaborators Jan 2, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

enhancement Requests to existing resources that expand the functionality or scope. service/lambda Issues and PRs that pertain to the lambda service. size/M Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

event_source_mapping support for batch windows of up to 5 minutes for functions with Amazon SQS

2 participants