Merge remote-tracking branch 'IQSS/develop' into IQSS/12020-Payara7_Java21_update

Author: qqmyers

.github/workflows/container_base_push.yml

@@ -50,7 +50,7 @@ jobs:
             # In case this is a push to develop, we care about buildtime.
             # Configure a remote ARM64 build host in addition to the local AMD64 in two steps.
             - name: Setup SSH agent
-              uses: webfactory/ssh-agent@v0.9.1
+              uses: webfactory/ssh-agent@v0.10.0
               with:
                   ssh-private-key: ${{ secrets.BUILDER_ARM64_SSH_PRIVATE_KEY }}
             - name: Provide the known hosts key and the builder config

doc/sphinx-guides/source/admin/metadatacustomization.rst

@@ -666,10 +666,6 @@ When creating new metadata blocks, please review the :doc:`/style/text` section
 
 If there are tips that you feel are omitted from this document, please open an issue at https://github.com/IQSS/dataverse/issues and consider making a pull request to make improvements. You can find this document at https://github.com/IQSS/dataverse/blob/develop/doc/sphinx-guides/source/admin/metadatacustomization.rst
 
-Alternatively, you are welcome to request "edit" access to this "Tips for Dataverse Software metadata blocks from the community" Google doc: https://docs.google.com/document/d/1XpblRw0v0SvV-Bq6njlN96WyHJ7tqG0WWejqBdl7hE0/edit?usp=sharing
-
-The thinking is that the tips can become issues and the issues can eventually be worked on as features to improve the Dataverse Software metadata system.
-
 Development Tasks Specific to Changing Fields in Core Metadata Blocks
 ---------------------------------------------------------------------
 

doc/sphinx-guides/source/api/native-api.rst

@@ -6424,7 +6424,7 @@ The fully expanded example above (without environment variables) looks like this
 Show Disclaimer for Publishing Datasets
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The setting "PublishDatasetDisclaimerText", when set, will prevent a draft dataset from being published through the UI without the user acknowledging the disclaimer.
+The setting :ref:`:PublishDatasetDisclaimerText`, when set, will prevent a draft dataset from being published through the UI without the user acknowledging the disclaimer.
 
 .. note:: See :ref:`show-custom-popup-for-publishing-datasets` if the user acknowledgment is not required but you want the message to be displayed in the UI.
 .. note:: See :ref:`curl-examples-and-environment-variables` if you are unfamiliar with the use of export below.

doc/sphinx-guides/source/installation/config.rst

@@ -4075,7 +4075,12 @@ dataverse.feature.only-update-datacite-when-needed
 
 Only contact DataCite to update a DOI after checking to see if DataCite has outdated information (for efficiency, lighter load on DataCite, especially when using file DOIs).
 
+.. _dataverse.feature.require-embargo-reason:
 
+dataverse.feature.require-embargo-reason
+++++++++++++++++++++++++++++++++++++++++
+
+Require an embargo reason when a user creates an embargo on one or more files. See :ref:`embargoes`.
 
 .. _:ApplicationServerSettings:
 
@@ -4826,6 +4831,10 @@ If you have a long text string, you can upload it as a file as in the example be
 
 ``curl -X PUT --upload-file /tmp/long.txt http://localhost:8080/api/admin/settings/:DatasetPublishPopupCustomText``
 
+There is a related setting called :ref:`:PublishDatasetDisclaimerText` that also makes text appear on the popup when publishing, but it requires a checkbox to be clicked.
+
+See also :ref:`show-custom-popup-for-publishing-datasets` in the API Guide.
+
 :DatasetPublishPopupCustomTextOnAllVersions
 +++++++++++++++++++++++++++++++++++++++++++
 
@@ -5356,6 +5365,10 @@ The text displayed to the user that must be acknowledged prior to publishing a D
 
 ``curl -X PUT -d "By publishing this dataset, I fully accept all legal responsibility for ensuring that the deposited content is: anonymized, free of copyright violations, and contains data that is computationally reusable. I understand and agree that any violation of these conditions may result in the immediate removal of the dataset by the repository without prior notice." http://localhost:8080/api/admin/settings/:PublishDatasetDisclaimerText``
 
+There is a similar setting called :ref:`:DatasetPublishPopupCustomText` that also makes text appear on the popup when publishing, but it is only informational. There is no checkbox to click.
+
+See also :ref:`show-disclaimer-for-publishing-datasets` in the API Guide.
+
 .. _:BagItHandlerEnabled:
 
 :BagItHandlerEnabled

doc/sphinx-guides/source/user/dataset-management.rst

@@ -750,7 +750,7 @@ Note that only one Preview URL (normal or with anonymized access) can be configu
 Embargoes
 =========
 
-A Dataverse instance may be configured to support file-level embargoes. Embargoes make file content inaccessible after a dataset version is published  - until the embargo end date. A reason for the embargo may be supplied when creating the embargo. A reason may be required in some Dataverse instances.
+A Dataverse instance may be configured to support file-level embargoes. Embargoes make file content inaccessible after a dataset version is published  - until the embargo end date. A reason for the embargo may be supplied when creating the embargo. A reason may be :ref:`required <dataverse.feature.require-embargo-reason>` in some Dataverse instances.
 This means that file previews and the ability to download files will be blocked. The effect is similar to when a file is restricted except that the embargo will end at the specified date without further action and during the embargo, requests for file access cannot be made. 
 Embargoes of files in a version 1.0 dataset may also affect the date shown in the dataset and file citations. The recommended practice is for the citation to reflect the date on which all embargoes on files in version 1.0 end. (Since Dataverse creates one persistent identifier per dataset and doesn't create new ones for each version, the publication of later versions, with or without embargoed files, does not affect the citation date.)
 
@@ -872,6 +872,8 @@ If your installation is configured to use DataCite as a persistent ID (PID) prov
 Review Datasets
 ---------------
 
+.. note:: This is an experimental feature.
+
 .. _review-datasets-overview:
 
 Review Dataset Overview

src/main/java/edu/harvard/iq/dataverse/DvObjectServiceBean.java

@@ -223,6 +223,10 @@ public DvObject updateContentIndexTime(DvObject dvObject) {
      * updateContentIndexTime method.
      */
     @TransactionAttribute(REQUIRES_NEW)
+    public DvObject updatePermissionIndexTimeInNewTransaction(DvObject dvObject) {
+        return updatePermissionIndexTime(dvObject);
+    }
+    
     public DvObject updatePermissionIndexTime(DvObject dvObject) {
         /**
          * @todo to avoid a possible OptimisticLockException, should we merge

src/main/java/edu/harvard/iq/dataverse/search/SolrIndexServiceBean.java

@@ -300,7 +300,12 @@ public IndexResponse indexPermissionsForOneDvObject(DvObject dvObject) {
             persistToSolr(docs);
             boolean updatePermissionTimeSuccessful = false;
             if (dvObject != null) {
-                DvObject savedDvObject = dvObjectService.updatePermissionIndexTime(dvObject);
+                DvObject savedDvObject = null;
+                if (dvObject.isInstanceofDataset()) {
+                    savedDvObject = dvObjectService.updatePermissionIndexTimeInNewTransaction(dvObject);
+                } else {
+                    savedDvObject = dvObjectService.updatePermissionIndexTime(dvObject);
+                }
                 if (savedDvObject != null) {
                     updatePermissionTimeSuccessful = true;
                 }

src/main/java/edu/harvard/iq/dataverse/util/bagit/BagGenerator.java

@@ -123,7 +123,6 @@ public class BagGenerator {
     private PoolingHttpClientConnectionManager cm = null;
 
     private ChecksumType hashtype = null;
-    private boolean ignorehashes = false;
 
     private long dataCount = 0l;
     private long totalDataSize = 0l;
@@ -158,6 +157,8 @@ public class BagGenerator {
     private boolean usingFetchFile = false;
     private boolean createHoleyBag = false;
     private List<FileEntry> oversizedFiles = new ArrayList<>();
+
+    private ChecksumType defaultHashtype;
 
     // Bag-info.txt field labels
     private static final String CONTACT_NAME = "Contact-Name: ";
@@ -244,6 +245,13 @@ public BagGenerator(jakarta.json.JsonObject oremapObject, String dataciteXml, Ma
             e.printStackTrace();
         }
         initializeHoleyBagLimits();
+        try {
+            // Use the current type if we can retrieve it
+            defaultHashtype = CDI.current().select(SystemConfig.class).get().getFileFixityChecksumAlgorithm();
+        } catch (Exception e) {
+            // Default to MD5 if we can't
+            defaultHashtype = DataFile.ChecksumType.MD5;
+        }
     }
 
     private void initializeHoleyBagLimits() {
@@ -255,10 +263,6 @@ private void initializeHoleyBagLimits() {
                     ", createHoleyBag: " + createHoleyBag);
     }
 
-    public void setIgnoreHashes(boolean val) {
-        ignorehashes = val;
-    }
-
     public static void println(String s) {
         System.out.println(s);
         System.out.flush();
@@ -353,13 +357,8 @@ public boolean generateBag(OutputStream outputStream) throws Exception {
             sha1StringBuffer.append(sha1Entry.getValue() + " " + path);
         }
         if(hashtype == null) { // No files - still want to send an empty manifest to nominally comply with BagIT specification requirement.
-            try {
-                // Use the current type if we can retrieve it
-                hashtype = CDI.current().select(SystemConfig.class).get().getFileFixityChecksumAlgorithm();
-            } catch (Exception e) {
-                // Default to MD5 if we can't
-                hashtype = DataFile.ChecksumType.MD5;
-            }
+                // Use the default
+                hashtype = defaultHashtype;
         }
         if (!(hashtype == null)) {
             String manifestName = "manifest-";
@@ -644,10 +643,6 @@ private void processAllFiles(List<FileEntry> sortedFiles)
         // Track titles to detect duplicates
         Set<String> titles = new HashSet<>();
 
-        if ((hashtype == null) | ignorehashes) {
-            hashtype = DataFile.ChecksumType.SHA512;
-        }
-        
         for (FileEntry entry : sortedFiles) {
             // Extract all needed information from the JsonObject reference
             JsonObject child = entry.jsonObject;
@@ -679,12 +674,15 @@ private void processAllFiles(List<FileEntry> sortedFiles)
                     childHash = child.getAsJsonObject(JsonLDTerm.checksum.getLabel()).get("@value").getAsString();
                 }
             }
-            
+            //Pick a hashtype if we encounter a file that doesn't have one
+            if (hashtype == null) {
+                hashtype = defaultHashtype;
+            }
             resourceUsed[entry.resourceIndex] = true;
             String dataUrl = entry.getDataUrl();
 
             try {
-                if ((childHash == null) | ignorehashes) {
+                if (childHash == null) {
                     // Generate missing hash
 
                     try (InputStream inputStream = getInputStreamSupplier(dataUrl).get()){