Part of #637 (Overly Complex Configuration File)
Is your feature request related to a problem? Please describe.
The current minio.crossStorage configuration is confusing and hard to understand for users:
-
The name describes the scenario, not the behavior. crossStorage sounds like "are you doing cross-storage backup?" but it actually controls how data is copied — via the storage system's native COPY API (false) or proxied through the backup server process (true). Users have to read a dedicated FAQ just to understand what it means (docs/user_guide/cross_storage.md).
-
Manual configuration is needed for same-type-different-instance scenarios. Auto-detection (core/backup/task.go:111) only compares Provider — if both are minio but at different addresses (Minio A → Minio B), the user must manually set crossStorage: true. This is the most confusing case: "both are Minio, why do I need crossStorage?"
-
Wrong config section. crossStorage is under the minio section, but it's not a MinIO setting — it's a backup behavior setting.
-
The FAQ in the docs is itself evidence of a naming failure. If a config option needs a dedicated FAQ entry explaining "what does this mean", the name isn't working.
Describe the solution you'd like.
This is a breaking change proposal:
1. Smarter auto-detection (P0) — eliminate manual configuration
Extend auto-detection to compare address and port in addition to provider:
// current: only compares provider
if backupProvider != milvusProvider { copyByServer = true }
// proposed: also compare address/port
if backupProvider != milvusProvider ||
backupAddress != milvusAddress ||
backupPort != milvusPort {
copyByServer = true
}
This covers the Minio A → Minio B scenario automatically. Users almost never need to set this manually.
2. Rename the config (P1) — name should describe behavior, not scenario
Replace minio.crossStorage with an enum-based copyMode:
backup:
copyMode: "auto" # auto-detect (default, recommended)
# copyMode: "direct" # force storage COPY API
# copyMode: "proxy" # force server-side proxy transfer
auto: current implicit behavior, picks the right mode automatically
direct: uses storage's native COPY API (fast, requires both ends accessible to each other)
proxy: data flows through backup server (slower, works across any storage combination)
3. Move config from minio to backup section (P2)
crossStorage is not a storage configuration — it's a backup operation behavior. It belongs under the backup section.
4. Log the actual copy mode at task start (P3)
INFO copy mode: proxy (auto-detected: backup and milvus storage have different addresses)
INFO copy mode: direct (same storage, using native COPY API)
Describe an alternate solution.
A minimal non-breaking approach: keep crossStorage as-is but improve auto-detection (P0) so users rarely encounter it. Add deprecation warnings pointing to the new config name.
Anything else? (Additional Context)
Code references
- Config definition:
internal/cfg/cfg.go:234,277
- Auto-detection:
core/backup/task.go:110-113, core/check/task.go:139-150
- Copier selection:
internal/storage/copier.go:126-137
- Restore skip logic:
core/restore/coll_task.go:336-343
- Backup copy:
core/backup/coll_dml_task.go:474-483
- User docs:
docs/user_guide/cross_storage.md
Migration path
- Support both old (
minio.crossStorage) and new config for one release cycle with a deprecation warning
- Remove the old config in the following release
Part of #637 (Overly Complex Configuration File)
Is your feature request related to a problem? Please describe.
The current
minio.crossStorageconfiguration is confusing and hard to understand for users:The name describes the scenario, not the behavior.
crossStoragesounds like "are you doing cross-storage backup?" but it actually controls how data is copied — via the storage system's native COPY API (false) or proxied through the backup server process (true). Users have to read a dedicated FAQ just to understand what it means (docs/user_guide/cross_storage.md).Manual configuration is needed for same-type-different-instance scenarios. Auto-detection (
core/backup/task.go:111) only comparesProvider— if both areminiobut at different addresses (Minio A → Minio B), the user must manually setcrossStorage: true. This is the most confusing case: "both are Minio, why do I need crossStorage?"Wrong config section.
crossStorageis under theminiosection, but it's not a MinIO setting — it's a backup behavior setting.The FAQ in the docs is itself evidence of a naming failure. If a config option needs a dedicated FAQ entry explaining "what does this mean", the name isn't working.
Describe the solution you'd like.
This is a breaking change proposal:
1. Smarter auto-detection (P0) — eliminate manual configuration
Extend auto-detection to compare
addressandportin addition toprovider:This covers the Minio A → Minio B scenario automatically. Users almost never need to set this manually.
2. Rename the config (P1) — name should describe behavior, not scenario
Replace
minio.crossStoragewith an enum-basedcopyMode:auto: current implicit behavior, picks the right mode automaticallydirect: uses storage's native COPY API (fast, requires both ends accessible to each other)proxy: data flows through backup server (slower, works across any storage combination)3. Move config from
miniotobackupsection (P2)crossStorageis not a storage configuration — it's a backup operation behavior. It belongs under thebackupsection.4. Log the actual copy mode at task start (P3)
Describe an alternate solution.
A minimal non-breaking approach: keep
crossStorageas-is but improve auto-detection (P0) so users rarely encounter it. Add deprecation warnings pointing to the new config name.Anything else? (Additional Context)
Code references
internal/cfg/cfg.go:234,277core/backup/task.go:110-113,core/check/task.go:139-150internal/storage/copier.go:126-137core/restore/coll_task.go:336-343core/backup/coll_dml_task.go:474-483docs/user_guide/cross_storage.mdMigration path
minio.crossStorage) and new config for one release cycle with a deprecation warning