Merge pull request #12110 from IQSS/12001-api-support-termofuse-guestbook

Feature Request: API to support Download Terms of Use and Guestbook

Author: sekmiller

doc/release-notes/12001-api-support-termofuse-guestbook.md

@@ -0,0 +1,25 @@
+## Feature Request: API to support Download Terms of Use and Guestbook
+
+## New Endpoints to download a file or files that required a Guestbook Response: POST
+A post to these endpoints with the body containing a JSON Guestbook Response will save the response and return a signed URL to download the file(s)
+
+`/api/access/datafile/{fileId:.+}`
+`/api/access/datafiles/{fileIds}`
+`/api/access/dataset/{id}`
+`/api/access/dataset/{id}/versions/{versionId}`
+
+A post to these endpoints with the body containing a JSON Guestbook Response will save the response before continuing the download.
+No signed URL option exists.
+`/api/access/datafiles`
+`/api/access/datafile/bundle/{fileId}` POST returns BundleDownloadInstance after processing Guestbook Responses from body.
+
+## New CRUD Endpoints for Guestbook:
+Create a Guestbook: POST `/api/guestbooks/{dataverseIdentifier}`
+Get a Guestbook: GET `/api/guestbooks/{id}`
+Get a list of Guestbooks linked to a Dataverse Collection: GET `/api/guestbooks/{dataverseIdentifier}/list`
+Enable/Disable a Guestbook: PUT `/api/guestbooks/{dataverseIdentifier}/{id}/enabled` Body: `true` or `false`
+Note: There is no Update or Delete at this time. You can disable a Guestbook and create a new one.
+
+## For Guestbook At Request:
+When JVM setting -Ddataverse.files.guestbook-at-request=true is used a request for access may require a Guestbook Response.
+PUT `/api/access/datafile/{id}/requestAccess` will now take a JSON Guestbook Response in the body.

doc/sphinx-guides/source/api/changelog.rst

@@ -7,6 +7,29 @@ This API changelog is experimental and we would love feedback on its usefulness.
     :local:
     :depth: 1
 
+v6.10
+-----
+- The following GET APIs will now return ``400`` if a required Guestbook Response is not supplied. A Guestbook Response can be passed to these APIs in the JSON body using a POST call. See the notes under :ref:`basic-file-access` and :ref:`download-by-dataset-by-version` for details.
+
+  - **/api/access/datafile/{fileId:.+}**
+
+  - **/api/access/datafiles/{fileIds}**
+
+  - **/api/access/dataset/{id}**
+
+  - **/api/access/dataset/{id}/versions/{versionId}**
+
+- The following POST APIs will now return ``400`` if a required Guestbook Response is not supplied. A Guestbook Response can be passed to these APIs in the JSON body. See the note under :ref:`basic-download-by-dataset` for details.
+
+  - **/api/access/datafiles**
+
+  - **/api/access/datafile/bundle/{fileId}**
+
+- The following PUT APIs will now return ``400`` if a required Guestbook Response is not supplied. When JVM setting -Ddataverse.files.guestbook-at-request=true is set a Guestbook Response may be required to be passed to these APIs in the JSON body. See the note under Configuration :ref:`dataverse.files.guestbook-at-request` for details.
+
+  - **/api/access/datafile/{id}/requestAccess**
+
+
 v6.9
 ----
 

doc/sphinx-guides/source/api/dataaccess.rst

@@ -27,11 +27,15 @@ Please note that in addition to the files from dataset, an additional file call
 
 There are two forms of the "download by dataset" API, a basic form and one that supports dataset versions.
 
+.. _basic-download-by-dataset:
+
 Basic Download By Dataset
 ~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The basic form downloads files from the latest accessible version of the dataset. If you are not using an API token, this means the most recently published version. If you are using an API token with full access to the dataset, this means the draft version or the most recently published version if no draft exists.
 
+.. note:: Files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url that can be used to download the file(s) via a browser or download manager. To determine what information is required in the response you must first get the Guestbook for the Dataset whose file(s) you are trying to access along with any Custom Questions. Call ``GET http://$SERVER/api/guestbooks/{id}`` with the guestbookId from the Dataset (See :ref:`dataset-json-representation` to get the Dataset with guestbookId) to retrieve the Guestbook and Custom Questions. Build the JSON response with the information requested and add it to the body of the POST call within "guestbookResponse":{}. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
 A curl example using a DOI (no version):
 
 .. code-block:: bash
@@ -48,6 +52,8 @@ The fully expanded example above (without environment variables) looks like this
 
   curl -L -O -J -H X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx https://demo.dataverse.org/api/access/dataset/:persistentId/?persistentId=doi:10.70122/FK2/N2XGBJ
 
+.. _download-by-dataset-by-version:
+
 Download By Dataset By Version
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -59,6 +65,8 @@ The second form of the "download by dataset" API allows you to specify which ver
 * ``x.y`` a specific version, where ``x`` is the major version number and ``y`` is the minor version number.
 * ``x`` same as ``x.0``
 
+.. note:: Files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url that can be used to download the file(s) via a browser or download manager. To determine what information is required in the response you must first get the Guestbook for the Dataset whose file(s) you are trying to access along with any Custom Questions. Call ``GET http://$SERVER/api/guestbooks/{id}`` with the guestbookId from the Dataset (See :ref:`dataset-json-representation` to get the Dataset with guestbookId) to retrieve the Guestbook and Custom Questions. Build the JSON response with the information requested and add it to the body of the POST call within "guestbookResponse":{}. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
 A curl example using a DOI (with version):
 
 .. code-block:: bash
@@ -78,6 +86,8 @@ The fully expanded example above (without environment variables) looks like this
 
   curl -O -J -H X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx https://demo.dataverse.org/api/access/dataset/:persistentId/versions/2.0?persistentId=doi:10.70122/FK2/N2XGBJ
 
+.. _basic-file-access:
+
 Basic File Access
 -----------------
 
@@ -91,6 +101,11 @@ Basic access URI:
 
     GET http://$SERVER/api/access/datafile/:persistentId?persistentId=doi:10.5072/FK2/J8SJZB
 
+.. note:: Files that require a Guestbook Response will require an additional step to supply the Guestbook Response. A POST to the same endpoint with the Guestbook Response in the body can return a signed url that can be used to download the file(s) via a browser or download manager. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide. In the following JSON example please note that the `name`, `email`, `institution`, and `position` fields will default to the User's account information if not included in the response. Call ``GET http://$SERVER/api/guestbooks/{id}`` with the guestbookId from the Dataset (See :ref:`dataset-json-representation` to get the Dataset with guestbookId) to retrieve the Guestbook and Custom Questions. Build the JSON response with the information requested and add it to the body of the POST call within "guestbookResponse":{}.
+
+  Example ::
+
+    POST http://$SERVER/api/access/datafile/:persistentId?persistentId=doi:10.5072/FK2/J8SJZB&signed=true -d '{"guestbookResponse": {"name": "My Name", "email": "myemail@example.com", "institution": "Harvard","position": "Staff", "answers": [{"id": 123,"value": "Good"},{"id": 124,"value": ["Multi","Line"]},{"id": 125,"value": "Yellow"}]}}'
 
 Parameters:
 ~~~~~~~~~~~
@@ -198,6 +213,8 @@ Returns the files listed, zipped. As of v6.7 the name of the zipped bundle will
 
 .. note:: If any of the datafiles have the ``DirectoryLabel`` attributes in the corresponding ``FileMetadata`` entries, these will be added as folders to the Zip archive, and the files will be placed in them accordingly. 
 
+.. note:: If Guestbook Responses are required they can be included in the body along with the file ids as JSON: ``{"fileIds" :[1,2,3], {"guestbookResponse": {"answers": []}}}``. For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
 Parameters: 
 ~~~~~~~~~~~
 
@@ -378,7 +395,9 @@ This method requests access to the datafile whose id is passed on the behalf of
 A curl example using an ``id``::
 
     curl -H "X-Dataverse-key:$API_TOKEN" -X PUT http://$SERVER/api/access/datafile/{id}/requestAccess
-    
+
+.. note:: Some installations of Dataverse may require you to provide a Guestbook Response when requesting access to certain restricted files. The response can be passed in the body of this call. See "Get a Guestbook for a Dataverse Collection" in the :doc:`native-api`.
+
 Grant File Access:
 ~~~~~~~~~~~~~~~~~~ 
 

doc/sphinx-guides/source/api/native-api.rst

@@ -1168,6 +1168,100 @@ The fully expanded example above (without environment variables) looks like this
 
   curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/dataverses/root/guestbookResponses?guestbookId=1" -o myResponses.csv
 
+.. _guestbook-api:
+
+Guestbooks
+~~~~~~~~~~
+
+Create a Guestbook for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Create a Guestbook that can be selected for a Dataset.
+You must have "EditDataverse" permission on the Dataverse collection.
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=root
+  export JSON='{"name": "my test guestbook","enabled": true,"emailRequired": true,"nameRequired": true,"institutionRequired": false,"positionRequired": false,"customQuestions": [{"question": "how is your day","required": true,"displayOrder": 0,"type": "text","hidden": false}]}'
+
+  curl -POST -H  "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}" -d "$JSON"
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -POST -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root" -d '{"name": "my test guestbook","enabled": true,"emailRequired": true,"nameRequired": true,"institutionRequired": false,"positionRequired": false}'
+
+Get a list of Guestbooks for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Get a list of Guestbooks for a Dataverse Collection
+You must have "EditDataverse" permission on the Dataverse collection.
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=root
+
+  curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}/list"`
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root/list"
+
+Get a Guestbook for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Get a Guestbook by its id
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export ID=1234
+
+  curl -H "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{ID}"`
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/1234"
+
+Enable or Disable a Guestbook for a Dataverse Collection
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For more about guestbooks, see :ref:`dataset-guestbooks` in the User Guide.
+
+Use this endpoint to enable or disable the Guestbook. A Guestbook can not be deleted or modified since there may be responses linked to it.
+You must have "EditDataverse" permission on the Dataverse collection.
+
+.. code-block:: bash
+
+  export API_TOKEN=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+  export SERVER_URL=https://demo.dataverse.org
+  export dataverseIdentifier=root
+  export ID=1234
+
+  curl -X PUT -d 'true' -H  "X-Dataverse-key:$API_TOKEN" "$SERVER_URL/api/guestbooks/{dataverseIdentifier}/{ID}/enabled"
+
+The fully expanded example above (without environment variables) looks like this:
+
+.. code-block:: bash
+
+  curl -X PUT -d 'true' -H "X-Dataverse-key:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" "https://demo.dataverse.org/api/guestbooks/root/1234"
+
 .. _collection-attributes-api:
 
 Change Collection Attributes
@@ -1774,6 +1868,8 @@ In all commands below, dataset versions can be referred to as:
 * ``x.y`` a specific version, where ``x`` is the major version number and ``y`` is the minor version number.
 * ``x`` same as ``x.0``
 
+.. _dataset-json-representation:
+
 Get JSON Representation of a Dataset
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 

docker-compose-dev.yml

@@ -60,6 +60,7 @@ services:
         -Ddataverse.pid.fake.label=FakeDOIProvider
         -Ddataverse.pid.fake.authority=10.5072
         -Ddataverse.pid.fake.shoulder=FK2/
+        #-Ddataverse.files.guestbook-at-request=true
         #-Ddataverse.lang.directory=/dv/lang
     ports:
       - "8080:8080" # HTTP (Dataverse Application)

scripts/api/data/guestbook-test-response.json

@@ -0,0 +1,17 @@
+{"guestbookResponse": {
+    "answers": [
+        {
+            "id": @QID1,
+            "value": "Good"
+        },
+        {
+            "id": @QID2,
+            "value": ["Multi","Line"]
+        },
+        {
+            "id": @QID3,
+            "value": "Yellow"
+        }
+    ]
+  }
+}

scripts/api/data/guestbook-test.json

@@ -0,0 +1,49 @@
+{
+  "name": "my test guestbook",
+  "enabled": true,
+  "emailRequired": true,
+  "nameRequired": true,
+  "institutionRequired": false,
+  "positionRequired": false,
+  "customQuestions": [
+    {
+      "question": "how's your day",
+      "required": true,
+      "displayOrder": 0,
+      "type": "text",
+      "hidden": false
+    },
+    {
+      "question": "Describe yourself",
+      "required": false,
+      "displayOrder": 1,
+      "type": "textarea",
+      "hidden": false
+    },
+    {
+      "question": "What color car do you drive",
+      "required": true,
+      "displayOrder": 2,
+      "type": "options",
+      "hidden": false,
+      "optionValues": [
+        {
+          "value": "Red",
+          "displayOrder": 0
+        },
+        {
+          "value": "White",
+          "displayOrder": 1
+        },
+        {
+          "value": "Yellow",
+          "displayOrder": 2
+        },
+        {
+          "value": "Purple",
+          "displayOrder": 3
+        }
+      ]
+    }
+  ]
+}

src/main/java/edu/harvard/iq/dataverse/CustomQuestion.java

@@ -1,9 +1,12 @@
 package edu.harvard.iq.dataverse;
-import java.io.Serializable;
-import java.util.List;
+
 import jakarta.persistence.*;
 import jakarta.validation.constraints.NotBlank;
 
+import java.io.Serializable;
+import java.util.List;
+import java.util.stream.Collectors;
+
 /**
  *
  * @author skraffmiller
@@ -92,6 +95,12 @@ public void setQuestionString(String questionString) {
     public List<CustomQuestionValue> getCustomQuestionValues() {
         return customQuestionValues;
     }
+
+    public List<String> getCustomQuestionOptions() {
+        return customQuestionValues.stream()
+                .map(CustomQuestionValue::getValueString)
+                .collect(Collectors.toList());
+    }
 
     public String getCustomQuestionValueString(){
         String retString = "";

src/main/java/edu/harvard/iq/dataverse/CustomQuestionResponse.java

@@ -5,11 +5,12 @@
  */
 package edu.harvard.iq.dataverse;
 
-import java.io.Serializable;
-import java.util.List;
 import jakarta.faces.model.SelectItem;
 import jakarta.persistence.*;
 
+import java.io.Serializable;
+import java.util.List;
+
 /**
  *
  * @author skraffmiller