Parent issue: #831
Feature Request / Improvement
Orphan cleanup (table/orphan_cleanup.go:428,469) and snapshot expiration (table/updates.go:540) delete files one at a time via IO.Remove. For cloud object stores this is inefficient — S3's DeleteObjects handles 1000 keys per call, and GCS/Azure have similar batch APIs.
An optional BulkRemovableIO interface should be added to io/io.go with a RemoveAll(paths []string) error method. When the IO implementation supports it, orphan cleanup and snapshot expiration would batch deletes instead of looping one file at a time.
Implementations that don't support bulk delete simply don't implement the interface and the existing behavior is unchanged.
Parent issue: #831
Feature Request / Improvement
Orphan cleanup (
table/orphan_cleanup.go:428,469) and snapshot expiration (table/updates.go:540) delete files one at a time viaIO.Remove. For cloud object stores this is inefficient — S3's DeleteObjects handles 1000 keys per call, and GCS/Azure have similar batch APIs.An optional
BulkRemovableIOinterface should be added toio/io.gowith aRemoveAll(paths []string) errormethod. When the IO implementation supports it, orphan cleanup and snapshot expiration would batch deletes instead of looping one file at a time.Implementations that don't support bulk delete simply don't implement the interface and the existing behavior is unchanged.