Centralize CSV parsing (CsvUtil) + CORS origin echo & Vary header improvements

Author: ErykKul

doc/release-notes/11744-cors-echo-origin-vary.md

@@ -1,62 +1,33 @@
-# 11744: CORS header handling fixes (echo single Origin, add Vary: Origin, multi-origin allow, sanitization)
+# 11744: CORS handling improvements
 
-This branch adjusts the CORS filter so browser clients work correctly when multiple origins are allowed.
+Modernizes CORS so browser integrations (previewers, external tools, JS clients) work correctly with multiple origins and proper caching.
 
-## What changed
-- Access-Control-Allow-Origin (ACAO) now echoes the single request `Origin` when it matches an allowlist from `dataverse.cors.origin`.
-- `Vary: Origin` is added when echoing a specific origin to keep caches correct across different origins.
-- Comma-separated origin lists are supported; surrounding quotes in CSV configs are stripped.
-- Sanitization is applied to CORS header lists (methods/allow/expose) to avoid quoted values that can break preflight checks.
-- Deprecated DB fallback for enabling CORS is removed; CORS is considered enabled only when `dataverse.cors.origin` is set as a JVM options/Microprofile setting.
+## Highlights
+* Echoes the request origin (`Access-Control-Allow-Origin`) when it matches `dataverse.cors.origin`.
+* Adds `Vary: Origin` for per-origin responses (not for wildcard).
+* Supports comma‑separated origin list; any `*` in the list = wildcard mode.
+* CORS now only enabled when `dataverse.cors.origin` is set (deprecated `:AllowCors` no longer enables it).
+* Sanitizes CORS CSV settings (`dataverse.cors.methods`, `dataverse.cors.headers.allow`, `dataverse.cors.headers.expose`).
+* Docs updated (Installation, Big Data Support, External Tools, File Previews); new tests cover edge cases.
 
-## Upgrade / run notes (non-SQL)
-To keep CORS working after pulling this branch:
+## Admin Action
+Set `dataverse.cors.origin` explicitly (required). Use explicit origins (not `*`) for credentialed requests. Ensure proxies keep `Vary: Origin`.
 
-1) Configure origins as JVM options/Microprofile settings (no quotes):
-- Single origin:
-  - `dataverse.cors.origin=https://example.org`
-- Multiple origins (comma-separated):
-  - `dataverse.cors.origin=https://libis.github.io,https://gdcc.github.io`
-- Wildcard:
-  - `dataverse.cors.origin=*`
-  - Note: Browsers reject `*` when credentialed requests are used (cookies/Authorization headers). Prefer explicit origins for those cases.
-
-2) Optional headers/methods lists (unquoted, comma-separated CSV):
-- `dataverse.cors.methods`
-- `dataverse.cors.headers.allow`
-- `dataverse.cors.headers.expose`
-
-Avoid surrounding values with quotes (e.g., do not use `"Accept, Content-Type"`). Quotes will be stripped but may cause confusion.
-
-3) If you previously relied on the database setting to enable CORS (deprecated `AllowCors`), set `dataverse.cors.origin` instead. The DB fallback is no longer used.
-
-4) Reverse proxies/caches: `Vary: Origin` is now emitted. Ensure your proxy does not drop this header.
-
-## Verify
-Preflight (replace DV_URL with your base URL):
-
-```bash
-curl -i -X OPTIONS \
-  -H "Origin: https://libis.github.io" \
-  -H "Access-Control-Request-Method: GET" \
-  "${DV_URL}/api/info/version"
+Examples:
 ```
-
-Expected:
-- `Access-Control-Allow-Origin: https://libis.github.io`
-- `Vary: Origin` present
-
-Actual request:
-
-```bash
-curl -i \
-  -H "Origin: https://libis.github.io" \
-  "${DV_URL}/api/info/version"
+dataverse.cors.origin=https://example.org
+dataverse.cors.origin=https://libis.github.io,https://gdcc.github.io
+dataverse.cors.origin=*
 ```
+Optional (unquoted):
+```
+dataverse.cors.methods=GET, POST, OPTIONS, PUT, DELETE
+```
+
+## Compatibility
+* Must configure `dataverse.cors.origin`; `:AllowCors` no longer sufficient.
+* Any `*` triggers wildcard (no per-origin echo / no Vary header).
 
-Expected:
-- Same ACAO echo as above
+## Docs
+See updated `dataverse.cors.origin` section and related notes in Big Data Support (S3), External Tools, and File Previews.
 
-## Backward compatibility
-- Instances relying on the deprecated DB-based CORS enablement must set `dataverse.cors.origin` to keep CORS enabled.
-- Quoted CORS configuration values may behave differently; remove quotes going forward.

doc/sphinx-guides/source/api/external-tools.rst

@@ -11,6 +11,9 @@ Introduction
 
 External tools are additional applications the user can access or open from your Dataverse installation to preview, explore, and manipulate data files and datasets. The term "external" is used to indicate that the tool is not part of the main Dataverse Software.
 
+.. note::
+  Browser-based preview or explore tools that make XHR/fetch calls back to the Dataverse API must have CORS explicitly enabled on the Dataverse installation via :ref:`dataverse.cors.origin <dataverse.cors.origin>`. The legacy ``:AllowCors`` database setting is deprecated and no longer enables CORS by itself. Be sure the origins hosting your tool (or ``*`` when appropriate) are included in ``dataverse.cors.origin``; otherwise requests from your tool will be blocked by the browser even if the tool itself loads correctly.
+
 Once you have created the external tool itself (which is most of the work!), you need to teach a Dataverse installation how to construct URLs that your tool needs to operate. For example, if you've deployed your tool to fabulousfiletool.com your tool might want the ID of a file and the siteUrl of the Dataverse installation like this: https://fabulousfiletool.com?fileId=42&siteUrl=https://demo.dataverse.org
 
 In short, you will be creating a manifest in JSON format that describes not only how to construct URLs for your tool, but also what types of files your tool operates on, where it should appear in the Dataverse installation web interfaces, etc. 

doc/sphinx-guides/source/developers/big-data-support.rst

@@ -57,6 +57,15 @@ Allow CORS for S3 Buckets
 **IMPORTANT:** One additional step that is required to enable direct uploads via a Dataverse installation and for direct download to work with previewers and direct upload to work with dvwebloader (:ref:`folder-upload`) is to allow cross site (CORS) requests on your S3 store.
 The example below shows how to enable CORS rules (to support upload and download) on a bucket using the AWS CLI command line tool. Note that you may want to limit the AllowedOrigins and/or AllowedHeaders further.  https://github.com/gdcc/dataverse-previewers/wiki/Using-Previewers-with-download-redirects-from-S3 has some additional information about doing this.
 
+Dataverse itself will only emit the necessary ``Access-Control-*`` headers to browsers when CORS has been explicitly enabled via the JVM/MicroProfile setting :ref:`dataverse.cors.origin <dataverse.cors.origin>`. The legacy database setting ``:AllowCors`` no longer turns CORS on. You must both:
+
+* Configure an appropriate ``dataverse.cors.origin`` value (single origin, comma-separated list, or ``*``) on the Dataverse application server; and
+* Configure a matching/compatible CORS policy on each S3 bucket (and any CDN/proxy in front of it) that will be used for direct upload or for redirect (download-redirect) operations consumed by previewers.
+
+If you specify multiple origins in ``dataverse.cors.origin`` Dataverse will echo back the requesting origin (when it matches) and will include ``Vary: Origin`` so that shared caches do not serve one origin's response to another. If you configure ``*`` Dataverse will respond with ``Access-Control-Allow-Origin: *`` (note that browsers will not allow credentialed requests with a wildcard).
+
+Make sure the bucket CORS configuration ``AllowedOrigins`` is at least as permissive as the origins you configure in ``dataverse.cors.origin``. If the bucket allows ``*`` but the Dataverse application only allows a subset, the browser will still enforce the more restrictive application response.
+
 If you'd like to check the CORS configuration on your bucket before making changes:
 
 ``aws s3api get-bucket-cors --bucket <BUCKET_NAME>``

doc/sphinx-guides/source/installation/config.rst

@@ -3667,17 +3667,22 @@ The following settings control Cross-Origin Resource Sharing (CORS) for your Dat
 dataverse.cors.origin
 +++++++++++++++++++++
 
-Allowed origins for CORS requests. The default with no value set is to not include CORS headers. However, if the deprecated :AllowCors setting is explicitly set to true the default is "\*" (all origins).
-When the :AllowsCors setting is not used, you must set this setting to "\*" or a list of origins to enable CORS headers.
+Allowed origins for CORS requests. If this setting is not defined, CORS headers are not added. Set to ``*`` to allow all origins (note that browsers will not allow credentialed requests with ``*``) or provide a comma-separated list of explicit origins.
 
-Multiple origins can be specified as a comma-separated list.
+Multiple origins can be specified as a comma-separated list (whitespace is ignored):
 
 Example:
 
 ``./asadmin create-jvm-options '-Ddataverse.cors.origin=https://example.com,https://subdomain.example.com'``
 
 Can also be set via any `supported MicroProfile Config API source`_, e.g. the environment variable ``DATAVERSE_CORS_ORIGIN``.
 
+Behavior:
+
+* When a list of origins is configured, Dataverse echoes the single matching request ``Origin`` value in ``Access-Control-Allow-Origin`` and adds ``Vary: Origin`` to support correct proxy/CDN caching.
+* When ``*`` is configured, ``Access-Control-Allow-Origin: *`` is sent and ``Vary`` is not modified.
+* The legacy database setting ``:AllowCors`` is deprecated and no longer enables CORS automatically; you must configure ``dataverse.cors.origin``.
+
 .. _dataverse.cors.methods:
 
 dataverse.cors.methods
@@ -4917,19 +4922,17 @@ This can be helpful in situations where multiple organizations are sharing one D
 or
 ``curl -X PUT -d '*' http://localhost:8080/api/admin/settings/:InheritParentRoleAssignments``
 
-:AllowCors (Deprecated)
-+++++++++++++++++++++++
+:AllowCors (Deprecated – no longer used once dataverse.cors.* settings exist)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 
 .. note::
-   This setting is deprecated. Please use the JVM settings above instead.
-   This legacy setting will only be used if the newer JVM settings are not set.
+  This legacy database setting has been superseded by the ``dataverse.cors.*`` JVM/MicroProfile settings. In current versions CORS is only enabled when ``dataverse.cors.origin`` is explicitly set. Existing values of ``:AllowCors`` are ignored if ``dataverse.cors.origin`` is unset.
 
-Enable or disable support for Cross-Origin Resource Sharing (CORS) by setting ``:AllowCors`` to ``true`` or ``false``.
+Historical behavior (prior versions) allowed setting ``:AllowCors`` to ``true``/``false``. Administrators should migrate to the JVM/MicroProfile setting:
 
-``curl -X PUT -d true http://localhost:8080/api/admin/settings/:AllowCors``
+``./asadmin create-jvm-options '-Ddataverse.cors.origin=*'``
 
-.. note::
-   New values for this setting will only be used after a server restart.
+or a comma-separated list of allowed origins.
 
 :ChronologicalDateFacets
 ++++++++++++++++++++++++

doc/sphinx-guides/source/user/dataset-management.rst

@@ -175,6 +175,9 @@ File Previews
 
 Dataverse installations can add previewers for common file types uploaded by their research communities. The previews appear on the file page. If a preview tool for a specific file type is available, the preview will be created and will display automatically, after terms have been agreed to or a guestbook entry has been made, if necessary. File previews are not available for restricted files unless they are being accessed using a Preview URL. See also :ref:`previewUrl`. When the dataset license is not the default license, users will be prompted to accept the license/data use agreement before the preview is shown. See also :ref:`license-terms`.
 
+.. note::
+   Some previewers run purely in the browser and make direct (JavaScript) requests back to the Dataverse API endpoints to retrieve file contents, metadata, or signed URLs. For these previewers to function when hosted on a different origin (e.g., a CDN or a separate previewer service), the Dataverse installation must have CORS enabled via :ref:`dataverse.cors.origin <dataverse.cors.origin>`. Administrators should configure the list of allowed origins to include the host serving the previewers. The deprecated ``:AllowCors`` database setting no longer enables CORS.
+
 Previewers are available for the following file types:
 
 - Text

src/main/java/edu/harvard/iq/dataverse/DatasetFieldServiceBean.java

@@ -52,6 +52,7 @@
 import org.apache.http.protocol.HttpContext;
 import org.apache.http.util.EntityUtils;
 import edu.harvard.iq.dataverse.settings.SettingsServiceBean;
+import edu.harvard.iq.dataverse.util.CsvUtil;
 
 /**
  *
@@ -853,12 +854,12 @@ public String getFieldLanguage(String languages, String localeCode) {
         // If the fields list of supported languages contains the current locale (e.g.
         // the lang of the UI, or the current metadata input/display lang (tbd)), use
         // that. Otherwise, return the first in the list
-        String[] langStrings = languages.split("\\s*,\\s*");
-        if (langStrings.length > 0) {
-            if (Arrays.asList(langStrings).contains(localeCode)) {
+        final List<String> langStrings = CsvUtil.split(languages);
+        if (!langStrings.isEmpty()) {
+            if (langStrings.contains(localeCode)) {
                 return localeCode;
             } else {
-                return langStrings[0];
+                return langStrings.get(0);
             }
         }
         return null;

src/main/java/edu/harvard/iq/dataverse/FileMetadata.java

@@ -48,6 +48,7 @@
 import edu.harvard.iq.dataverse.datavariable.DataVariable;
 import edu.harvard.iq.dataverse.datavariable.VarGroup;
 import edu.harvard.iq.dataverse.datavariable.VariableMetadata;
+import edu.harvard.iq.dataverse.util.CsvUtil;
 import edu.harvard.iq.dataverse.util.DateUtil;
 import edu.harvard.iq.dataverse.util.StringUtil;
 import java.util.HashSet;
@@ -609,7 +610,7 @@ public int compare(FileMetadata o1, FileMetadata o2) {
     public static void setCategorySortOrder(String categories) {
        categoryMap=new HashMap<String, Long>();
        long i=1;
-       for(String cat: categories.split(",\\s*")) {
+       for(String cat: CsvUtil.split(categories)) {
            categoryMap.put(cat.toUpperCase(), i);
            i++;
        }

src/main/java/edu/harvard/iq/dataverse/SettingsWrapper.java

@@ -14,7 +14,7 @@
 import edu.harvard.iq.dataverse.settings.SettingsServiceBean;
 import edu.harvard.iq.dataverse.settings.SettingsServiceBean.Key;
 import edu.harvard.iq.dataverse.util.BundleUtil;
-import edu.harvard.iq.dataverse.util.MailUtil;
+import edu.harvard.iq.dataverse.util.CsvUtil;
 import edu.harvard.iq.dataverse.util.StringUtil;
 import edu.harvard.iq.dataverse.util.SystemConfig;
 import edu.harvard.iq.dataverse.UserNotification.Type;
@@ -396,7 +396,7 @@ public boolean isRsyncOnly() {
                 if (uploadMethods==null){
                     rsyncOnly = false;
                 } else {
-                    rsyncOnly = Arrays.asList(uploadMethods.toLowerCase().split("\\s*,\\s*")).size() == 1 && uploadMethods.toLowerCase().equals(SystemConfig.FileUploadMethods.RSYNC.toString());
+                    rsyncOnly = CsvUtil.split(uploadMethods).size() == 1 && uploadMethods.toLowerCase().equals(SystemConfig.FileUploadMethods.RSYNC.toString());
                 }
             }
         }
@@ -428,7 +428,7 @@ public Integer getUploadMethodsCount() {
             if (uploadMethods==null){
                 uploadMethodsCount = 0;
             } else {
-                uploadMethodsCount = Arrays.asList(uploadMethods.toLowerCase().split("\\s*,\\s*")).size();
+                uploadMethodsCount = CsvUtil.split(uploadMethods).size();
             } 
         }
         return uploadMethodsCount;
@@ -502,7 +502,8 @@ public boolean shouldBeAnonymized(DatasetField df) {
         if (anonymizedFieldTypes == null) {
             anonymizedFieldTypes = new ArrayList<String>();
             String names = get(SettingsServiceBean.Key.AnonymizedFieldTypeNames.toString(), "");
-            anonymizedFieldTypes.addAll(Arrays.asList(names.split(",\\s")));
+            // Use CsvUtil for consistent CSV parsing instead of raw regex split
+            anonymizedFieldTypes.addAll(CsvUtil.split(names));
         }
         return anonymizedFieldTypes.contains(df.getDatasetFieldType().getName());
     }
@@ -830,7 +831,7 @@ private Boolean getUploadMethodAvailable(String method){
         if (uploadMethods==null){
             return false;
         } else {
-           return  Arrays.asList(uploadMethods.toLowerCase().split("\\s*,\\s*")).contains(method);
+           return CsvUtil.splitToLowerCaseSet(uploadMethods).contains(method);
         }
     }
 

src/main/java/edu/harvard/iq/dataverse/api/Admin.java

@@ -111,6 +111,7 @@
 import edu.harvard.iq.dataverse.userdata.UserListResult;
 import edu.harvard.iq.dataverse.util.ArchiverUtil;
 import edu.harvard.iq.dataverse.util.BundleUtil;
+import edu.harvard.iq.dataverse.util.CsvUtil;
 import edu.harvard.iq.dataverse.util.FileUtil;
 import edu.harvard.iq.dataverse.util.SystemConfig;
 import edu.harvard.iq.dataverse.util.URLTokenUtil;
@@ -2167,7 +2168,7 @@ public Response addRoleAssignementsToChildren(@Context ContainerRequestContext c
         boolean inheritAllRoles = false;
         String rolesString = settingsSvc.getValueForKey(SettingsServiceBean.Key.InheritParentRoleAssignments, "");
         if (rolesString.length() > 0) {
-            ArrayList<String> rolesToInherit = new ArrayList<String>(Arrays.asList(rolesString.split("\\s*,\\s*")));
+            ArrayList<String> rolesToInherit = new ArrayList<>(CsvUtil.split(rolesString));
             if (!rolesToInherit.isEmpty()) {
                 if (rolesToInherit.contains("*")) {
                     inheritAllRoles = true;

src/main/java/edu/harvard/iq/dataverse/api/Datasets.java

@@ -5222,7 +5222,8 @@ public Response getPrivateUrlDatasetVersion(@PathParam("privateUrlToken") String
         }
         JsonObjectBuilder responseJson;
         if (isAnonymizedAccess) {
-            List<String> anonymizedFieldTypeNamesList = new ArrayList<>(Arrays.asList(anonymizedFieldTypeNames.split(",\\s")));
+            // Use CsvUtil for consistent CSV parsing
+            List<String> anonymizedFieldTypeNamesList = new ArrayList<>(CsvUtil.split(anonymizedFieldTypeNames));
             responseJson = json(dsv, anonymizedFieldTypeNamesList, true, returnOwners);
         } else {
             responseJson = json(dsv, null, true, returnOwners);
@@ -5248,7 +5249,8 @@ public Response getPreviewUrlDatasetVersion(@PathParam("previewUrlToken") String
         }
         JsonObjectBuilder responseJson;
         if (isAnonymizedAccess) {
-            List<String> anonymizedFieldTypeNamesList = new ArrayList<>(Arrays.asList(anonymizedFieldTypeNames.split(",\\s")));
+            // Use CsvUtil for consistent CSV parsing
+            List<String> anonymizedFieldTypeNamesList = new ArrayList<>(CsvUtil.split(anonymizedFieldTypeNames));
             responseJson = json(dsv, anonymizedFieldTypeNamesList, true, returnOwners);
         } else {
             responseJson = json(dsv, null, true, returnOwners);