Skip to content

Datasets with large number of files failed to Re-index v6.4 #11284

@bikramj

Description

@bikramj

What steps does it take to reproduce the issue?
We recently upgraded to v6.4 and Re-indexing got stuck 3-4 times on 2 different datasets which are really huge. We run re-index on a separate server which is not part of Production workload. While re-indexing was stuck Payara server got stuck too and app was unavailable. It ran for hours and 1 time Payara server crashed and restarted with OutofMemory error and other time Payara server did not come back after crash.
I had to manually set indextime of these 2 datasets in DB to proceed with Re-index of other datasets. Last time on 6.2 upgrade it took a while to re-index but we did not run into issue with frozen server.

First dataset has 1057 files with size 321G on disk
Second dataset 2872 files with 84G size

  • When does this issue occur?
    while Re-indexing bug datasets

  • Which page(s) does it occurs on?

  • What happens?
    Payara server hangs and stop responding

  • To whom does it occur (all users, curators, superusers)?
    All users

  • What did you expect to happen?
    Complete the re-indexing

Which version of Dataverse are you using?
6.4

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions