Skip to content
33 changes: 33 additions & 0 deletions storage/doc.go
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,39 @@ apply to single-shot uploads when user-provided checksum is provided.

Automatic checksumming can be disabled using [Writer.DisableAutoChecksum].

# Parallel Uploads

The parallel upload feature splits a large object into multiple parts and uploads them
in parallel. It is supported exclusively for gRPC clients. If used with a JSON
client, the configuration is ignored and a standard upload is performed.

Parallel uploads can yield higher throughput when uploading large objects.
However, there are several things which must be kept in mind when choosing to
use this strategy:
- Performing parallel uploads may incur additional costs. Class A
operations are performed to create each part. If a storage
class other than STANDARD is used, early deletion fees apply to deletion of
the parts.
- The service account/credentials used to perform the parallel
upload require `storage.objects.delete` in order to clean up the temporary
part objects.
- A failed upload can leave part objects behind
which will count as storage usage, and you will be billed for it.
Upon completion or failure of a parallel upload, the Writer makes a
best-effort attempt to clean up any temporary parts created. However, if the
program crashes there is no means for the client to perform the cleanup.
Temporary parts have the prefix: "gcs-go-sdk-pu-tmp". It is recommended to
set appropriate bucket lifecycle policies to reliably clean up any leftover
objects to avoid unnecessary storage costs.
- Using parallel uploads is not a one-size-fits-all solution.
They introduce overhead that is only offset when uploading
sufficiently large objects. The optimal threshold depends on many
factors; therefore, you should experiment with your specific
workload to determine if parallel uploads provide a benefit.

**Note:** This feature is currently experimental and its API surface may change
in future releases. It is not yet recommended for production use.

# Storage Control API

Certain control plane and long-running operations for Cloud Storage (including Folder
Expand Down
6 changes: 3 additions & 3 deletions storage/writer.go
Original file line number Diff line number Diff line change
Expand Up @@ -183,9 +183,9 @@ type Writer struct {
// parallel. Supported exclusively for gRPC clients. If used with a JSON
// client, the configuration is ignored and a standard upload is performed.
//
// Upon completion of a parallel upload, the Writer makes a best-effort attempt to clean up any temporary parts created.
// It is recommended to set appropriate bucket lifecycle policies to reliably clean up any leftover objects to avoid unnecessary storage costs.
// Temporary parts have the prefix: "gcs-go-sdk-pu-tmp".
// Parallel uploads can yield higher throughput when uploading large objects,
// but there are several considerations and trade-offs. Please refer to
// the [Parallel Uploads] section in the package documentation for details.
//
// **Note:** This feature is currently experimental and its API surface may change
// in future releases. It is not yet recommended for production use.
Expand Down
Loading