Skip to content

enhance: Client-side direct read for sealed segments from Object Storage #87

@tinswzy

Description

@tinswzy

Background
In Woodpecker, sealed segments are immutable and have already been fully uploaded to object storage (e.g. S3).
Currently, even for catch-up reads that only target sealed segments, clients still need to go through quorum nodes to read data.

This introduces unnecessary:

  • network hops
  • quorum node load
  • tail latency
    For sealed segments, quorum involvement is no longer required for correctness.

Proposal
Introduce a new read path: Clients can directly read sealed segment data from Object Storage, bypassing quorum nodes entirely.

Specifically:

  • For reads that only involve sealed segments
  • Client fetches segment metadata (location, offsets, checksums)
  • Client directly issues GET / range GET requests to Object Storage (S3)
  • Quorum nodes are skipped for data plane

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions