Skip to content

Fix client download rate calculations #2321

@bbockelm

Description

@bbockelm

While investigating a few strange download rate reports, I found some nasty bugs:

  • Starting with v7.16.0, the rate updates were done compared to the start of the transfer, not since the last rate update. That means the calculated rates would inevitably go down to zero and the transfer always times out after about 2 minutes.
  • The Client.MinimumDownloadSpeed parameter was always overwritten with the default value (102.4KB/s) or MInimumDownloadSpeed. This makes it impossible to set Client.MinimumDownloadSpeed directly.
  • The EWMA library we use assumes an update interval of 1 second yet we were updating it every 15 seconds. That means the transfer concurrency we use (which adjusts the minimum download speed lower if we are transferring multiple files) is an average over 7.5 minutes. We need to explicitly set the age (since the default of 30 has special semantics) and update once a second.

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingclientIssue affecting the OSDF clientcreate-patchPatch this into multiple versions of PelicancriticalHigh priority for next release

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions