Add an optional retry policy for HTTP(S) operations in fileio so transient network errors don’t immediately fail a Reader / Writer.
Motivation
Right now file read/write and HTTP GET/POST in FileIO are “one shot”: any temporary error (timeouts, 5xx, connection reset, etc.) aborts the operation. For long-running or unstable links, a built-in retry mechanism would make the API much more robust.
Proposed changes
-
Extend ReaderConfig / WriterConfig with a RetryPolicy:
enum class BackoffStrategy { none, linear, exponential };
struct RetryPolicy {
std::size_t maxAttempts = 1; // 1 = no retry (current behavior)
std::uint64_t initialDelayNs = 0; // delay before first retry
std::uint64_t maxDelayNs = 0; // backoff
BackoffStrategy backoff = BackoffStrategy::none;
std::vector<long> retryableStatusCodes; // e.g. {408, 429, 500, 502, 503, 504}
std::function<bool(long httpStatus, cpr::ErrorCode ec)> shouldRetry; // optional override
};
-
Add std::optional<RetryPolicy> retry; to ReaderConfig and WriterConfig.
-
Implement retry loop in
-
Default behavior: maxAttempts = 1 ⇒ no retries (preserve current semantics).
Notes / Open questions
- How to combine with
longPolling for GET? (Probably no retries there, or retries only on initial connect.)
- Emscripten: respect main-thread constraints while sleeping between retries.
Add an optional retry policy for HTTP(S) operations in
fileioso transient network errors don’t immediately fail aReader/Writer.Motivation
Right now file read/write and HTTP GET/POST in
FileIOare “one shot”: any temporary error (timeouts, 5xx, connection reset, etc.) aborts the operation. For long-running or unstable links, a built-in retry mechanism would make the API much more robust.Proposed changes
Extend
ReaderConfig/WriterConfigwith aRetryPolicy:Add
std::optional<RetryPolicy> retry;toReaderConfigandWriterConfig.Implement retry loop in
Default behavior:
maxAttempts = 1⇒ no retries (preserve current semantics).Notes / Open questions
longPollingfor GET? (Probably no retries there, or retries only on initial connect.)