feature/s3/transfermanager minimum part size should be 5MB

[jfantinhardesty]

Acknowledgements

• I have searched (https://github.com/aws/aws-sdk/issues?q=is%3Aissue) for past instances of this issue
• I have verified all of my SDK modules are up-to-date (you can perform a bulk update with go get -u github.com/aws/aws-sdk-go-v2/...)

Describe the bug

In the newly released S3 transfer manager, when trying to initiate a multipart upload with a part size less than 8MB gives an error "unable to initialize upload: part size must be at least 8388608 bytes"

Regression Issue

• Select this option if this issue appears to be a regression.

Expected Behavior

The S3 transfer manager should allow part sizes of 5MB or more as per the S3 spec.

Current Behavior

Unable to upload the object and gives error: "unable to initialize upload: part size must be at least 8388608 bytes"

Reproduction Steps

Upload an object with S3 transfer manager and set the partSize to 5MB.

mgr = transfermanager.New(cl.AwsS3Client, func(o *transfermanager.Options) {
	o.PartSizeBytes = 5 * 1024 * 1024
})

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

github.com/aws/aws-sdk-go-v2 v1.41.1
github.com/aws/aws-sdk-go-v2/feature/s3/transfermanager v0.1.1
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0

Compiler and Version used

1.25.6

Operating System and version

Ubuntu 24.04

[wty-Bryant]

Hi, the default 8MB part size is determined by the latest internal design specification. What's your use case that requires a multipart upload with 5MB part size?

The S3 transfer manager should allow part sizes of 5MB or more as per the S3 spec.

Also can you provide the source of that, since 5mb part size from s3 side is not the expected behavior, which was observed with some weird performance before. S3 best practices actually recommend between 8mb and 16mb.

[jfantinhardesty]

We have some existing workloads which will sometimes use the smallest possible part size of 5 MiB. As shown here https://docs.aws.amazon.com/AmazonS3/latest/userguide/qfacts.html the smallest allowable part size is 5 MiB. The current S3 manager allows setting the part size to 5 MiB, but the new transfer manager does not allow part sizes below 8 MiB.

[wty-Bryant]

We discussed internally and confirm this min part size enforcement on tmv2 is unnecessary. In the fix PR above, instead of changing the threshold, what we actually do is just remove the enforcement and leave the part size check to s3 server side, which should resolve your issue.
Will close #3297

[github-actions]

This issue is now closed. Comments on closed issues are hard for our team to see.
If you need more assistance, please open a new issue that references this one.