Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 23 additions & 3 deletions storage/writer.go
Original file line number Diff line number Diff line change
Expand Up @@ -183,9 +183,29 @@ type Writer struct {
// parallel. Supported exclusively for gRPC clients. If used with a JSON
// client, the configuration is ignored and a standard upload is performed.
//
// Upon completion of a parallel upload, the Writer makes a best-effort attempt to clean up any temporary parts created.
// It is recommended to set appropriate bucket lifecycle policies to reliably clean up any leftover objects to avoid unnecessary storage costs.
// Temporary parts have the prefix: "gcs-go-sdk-pu-tmp".
// Parallel uploads can yield higher throughput when uploading
// large objects. However, there are some things which must be kept in mind
// when choosing to use this strategy:
// - Performing parallel uploads costs more money. Class A operations
// are performed to create each part. If a storage
// tier other than STANDARD is used, early deletion fees apply to deletion of
// the parts.
// - The service account/credentials used to perform the parallel
// upload require `storage.objects.delete` in order to clean up the temporary
// part objects.
// - A failed upload can leave part objects behind
// which will count as storage usage, and you will be billed for it.
// Upon completion or failure of a parallel upload, the Writer makes a
// best-effort attempt to clean up any temporary parts created. However, if the
// program crashes there is no means for the client to perform the cleanup.
// Temporary parts have the prefix: "gcs-go-sdk-pu-tmp". It is recommended to
// set appropriate bucket lifecycle policies to reliably clean up any leftover
// objects to avoid unnecessary storage costs.
// - Using parallel uploads is not a one size fits all solution.
// They have very real overhead until uploading a large enough object. The
// inflection point is dependent upon many factors, and there is no one size
// fits all value. You will need to experiment with your deployment and workload
// to determine if parallel uploads are useful to you.
//
// **Note:** This feature is currently experimental and its API surface may change
// in future releases. It is not yet recommended for production use.
Expand Down