[Bug]: milvus-backup does not support backing up from local storage to MinIO for Milvus standalone？

[QunLeLZ]

Milvus-Backup Version

v0.5.11

Can you reproduce this issue on the latest version?

Not tested on the latest version

Current Behavior

I am using Milvus standalone 2.5.19 with local storage.
When I try to use milvus-backup to back up data from local storage and restore it to a new Milvus cluster with MinIO, milvus-backup does not support backing up local data to MinIO.

Expected Behavior

No response

Steps To Reproduce

Environment

Milvus version: 2.5.19 (standalone)
milvus-backup version: v0.5.11

Anything else?

No response

[huanghaoyuanhhy]

Hi @QunLeLZ, we'd like to support this scenario. Could you share the exact error message/logs and your backup.yaml config (with credentials redacted)? This will help us implement and verify this capability.

[QunLeLZ]

Thanks for your reply!
I configured backup.yaml like this:

# Configures the system log output.
log:
  level: info # Only supports debug, info, warn, error, panic, or fatal. Default 'info'.
  console: true # whether print log to console
  file:
    filename: "logs/backup.log"

# milvus proxy address, compatible to milvus.yaml
milvus:
  address: xxxxxxxx
  port: 31953
  user: ""
  password: ""

  # tls mode values [0, 1, 2]
  # 0 is close, 1 is one-way authentication, 2 is mutual authentication
  tlsMode: 0
  # tls cert path for validate server, will be used when tlsMode is 1 or 2
  caCertPath: ""
  serverName: ""
  # mutual tls cert path, for server to validate client.
  # Will be used when tlsMode is 2
  # for backward compatibility, if not set, will use tlsmode 1.
  # WARN: in future version, if user set tlsmode 2, but not set mtlsCertPath, will cause error.
  mtlsCertPath: ""
  mtlsKeyPath: ""

  # Milvus replicate msg channel name, default is by-dev-replicate-msg
  rpcChannelName: "by-dev-replicate-msg"

# Related configuration of minio, which is responsible for data persistence for Milvus.
minio:
  storageType: "local"
  rootPath: "/opt/data/hdd/misc/milvus/data/"

  # Backup storage configs, the storage you want to put the backup data
  backupStorageType: "minio" # support storage type: local, minio, s3, aws, gcp, ali(aliyun), azure, tc(tencent)
  backupAddress: "xxx" # Address of MinIO/S3
  backupRegion: "xxx"   # region of MinIO/S3
  backupPort: xxx  # Port of MinIO/S3
  backupAccessKeyID: "xxx"  # accessKeyID of MinIO/S3
  backupSecretAccessKey: "xxx" # MinIO/S3 encryption string
  # backupToken: ""       # token of MinIO/S3
  # backupGcpCredentialJSON: "/path/to/json-key-file" # The JSON content contains the gcs service account credentials.
  # Used only for the "gcpnative" cloud provider.
  backupBucketName: "test-bucket" # Bucket name to store backup data. Backup data will store to backupBucketName/backupRootPath
  backupRootPath: "backup" # Rootpath to store backup data. Backup data will store to backupBucketName/backupRootPath
  backupUseSSL: false # Access to MinIO/S3 with SSL

  # If you need to back up or restore data between two different storage systems, direct client-side copying is not supported.
  # Set this option to true to enable data transfer through Milvus Backup.
  # Note: This option will be automatically set to true if `minio.storageType` and `minio.backupStorageType` differ.
  # However, if they are the same but belong to different services, you must manually set this option to `true`.
  crossStorage: true

  # File size threshold (MiB) above which multipart copy is used for S3-compatible storage.
  # Default is 500. GCP does not support multipart copy and will always use single copy.
  multipartCopyThresholdMiB: 500
backup:
  parallelism:
    # number of threads to copy data. reduce it if blocks your storage's network bandwidth
    copydata: 128

    # number of collections to backup in parallel.
    backupCollection: 4
    # number of segments to backup in parallel
    backupSegment: 1024

    # Collection level parallelism to restore
    restoreCollection: 2
    # max number of import job to run in parallel,
    # should be less than milvus's config dataCoord.import.maxImportJobNum
    importJob: 768

  # keep temporary files during restore, only use to debug 
  keepTempFiles: false

  # Pause GC during backup through Milvus Http API. 
  gcPause:
    enable: true
    address: http://xxxxxx:30247

When I run ./milvus-backup check, it shows:

Milvus version: 2.5.19
Storage:
  milvus-storage-type: local
  milvus-bucket: a-bucket
  milvus-rootpath: opt/data/hdd/misc/milvus/data/
  backup-storage-type: minio
  backup-bucket: test-bucket
  backup-rootpath: backup

!!! Milvus root path is empty !!!
If your Milvus instance is expected to have data,
please check your minio configuration.
(address / bucket / rootPath).

Additional Observation:
Even though the ./milvus-backup check command throws the above error, when I actually run the backup command (milvus-backup backup), the meta data is successfully stored in MinIO. Only the actual data files (not meta) fail to be uploaded to MinIO, which means the meta data part works but the core data migration doesn't.

I think I wrote the configuration incorrectly.
I also tried removing the Milvus storage configs under the minio section,
but then I got this error:

Error: check: check milvus storage check: list milvus root dir storage: minio list prefix
The request signature we calculated does not match the signature you provided.
Check your key and signing method.

However, I haven't found any other configuration options that can support my use case — backing up data from Milvus standalone (local storage) to MinIO for restoration to a Milvus cluster.
Could you please help me check where I went wrong, or confirm if there is a correct configuration format for this cross-storage backup scenario?
Thank you very much!