Uniquedownloads API inconsistent with file downloads count on dataverse

[emily-katz]

What steps does it take to reproduce the issue?

There is a dataset: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DJVMTV that has 22 files and shows through dataverse 1,044 downloads, but when I ran the uniquedownloads API call, it returned 6 unique downloads.

• When does this issue occur?

  With all datasets

• Which page(s) does it occurs on?

  Download count listed on each dataset page and the return value of:
  With 'Make Data Count' it showed this dataset was downloaded over 23 times, which is more consistent with the file download count.

• What happens?

  counts don't make sense.

• To whom does it occur (all users, curators, superusers)?

  all users

• What did you expect to happen?

  I would expect if there have been 1000 file downloads on a dataset that has 22 files, that the total number of times the entire dataset would have been downloaded would be ~45 times. Even if we assume that some of these users downloaded this dataset more than 1 time, I think it is highly unlikely that 6 unique users each downloaded this dataset over 166 times.

Which version of Dataverse are you using?

6.6

Any related open or closed issues to this bug report?
No

Some discussion in Slack:

API call to get unique download counts:

For one of those datasets, the dataset page shows 1,044 downloads, while the API call for unique downloads returns 7

[qqmyers]

I thought we discussed this somewhere but I can't find it (I did see a slack message that I didn't respond to - I apologize for that.)

In any case, uniquedownloads are not very useful for publicly available data as 'unauthenticated user' is 1 user. In this case, I'd guess most people have downloaded the file(s) without logging in, and thus they all count as 1 unique user/ 1 uniquedownload.

FWIW: The uniquedownloads endpoint was developed at QDR to track downloads of restricted files/where login is required and was primarily intended to help equalize counts between datasets with few files and those with many. 1 download by 1 person of all files in a dataset results in n download counts (so the dataset with more files looks more popular), while the uniquedownloads count would be 1 in both cases. Similarly, if someone has a script that downloads the files again for each new analysis, versus downloading them once and reusing them, the downloads counts are inflated for the first case where the uniquedownloads would be the same in both cases.

I'm not sure what could be done to improve how /uniquedownloads works (without keeping the IP addresses of all non-logged in users in the database, which would be invasive and still limited in that users may have more than one IP address (or vice versa).

The MakeDataCounts metrics, which Harvard is planning to turn on, have another version of unique downloads that may be more relevant in this case - it just counts multiple downloads of the same file by the same user over a short time period as 1 unique count. That would still cause datasets with many files to have more MDC unique counts than smaller ones (or than the same dataset with a zipped file instead of multiple files), but it does distinguish between different unauthenticated/not-logged-in users.

It might make sense to make the limits of the uniquedownloads endpoint clearer in the guides/suggest that it's only helpful to track use by logged-in users.

[qqmyers]

FWIW: In the database, downloads are tracked by file and by 'session', so perhaps there is some way to get a sense of whether 1044 users have each downloaded one file, or 50 downloaded all 22, etc. A given (unauthenticated) user would have different sessions if they used different browser or downloaded files on subsequent days (or downloaded some files today and then downloaded others tomorrow), but perhaps knowing the number of sessions in which someone downloaded one or more files from the dataset could be a useful metric.

[cmbz]

@scolapasta adding this to the Dataverse backlog queue for prioritization

[cmbz]

2025-07-29

• As discussed in this comment, this discrepancy will be resolved once we implement the following:
  • Set up showing MDC metrics on dataset page for Harvard Dataverse dataverse.harvard.edu#385
  • Show MDC metrics in SPA Dataset page dataverse-frontend#779
  • An additional dependency is fully processing dataset backlogs using MDC
• I am leaving it open and on hold to track. Because MDC is a GREI objective, the work possibly may be accomplished in that timeline.