Skip to content

Commit e6b2661

Browse files
pdurbinpdurbin
authored
Merge pull request #10331 from Recherche-Data-Gouv/10316_cvoc_http_headers
CVOC : add HTTP request headers (Ontoportal integration)
2 parents 4678330 + 776fb38 commit e6b2661

9 files changed

Lines changed: 34 additions & 6 deletions

File tree

‎doc/release-notes/10316_cvoc_http_headers.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
You are now able to add HTTP request headers required by the External Vocabulary Services you are implementing.
2+
3+
A combined documentation can be found on pull request [#10404](https://github.com/IQSS/dataverse/pull/10404).
4+
5+
For more information, see issue [#10316](https://github.com/IQSS/dataverse/issues/10316) and pull request [gddc/dataverse-external-vocab-support#19](https://github.com/gdcc/dataverse-external-vocab-support/pull/19).

‎doc/sphinx-guides/source/admin/metadatacustomization.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -552,6 +552,8 @@ Great care must be taken when reloading a metadata block. Matching is done on fi
552552

553553
The ability to reload metadata blocks means that SQL update scripts don't need to be written for these changes. See also the :doc:`/developers/sql-upgrade-scripts` section of the Developer Guide.
554554

555+
.. _using-external-vocabulary-services:
556+
555557
Using External Vocabulary Services
556558
----------------------------------
557559

@@ -577,9 +579,9 @@ In general, the external vocabulary support mechanism may be a better choice for
577579
The specifics of the user interface for entering/selecting a vocabulary term and how that term is then displayed are managed by third-party Javascripts. The initial Javascripts that have been created provide auto-completion, displaying a list of choices that match what the user has typed so far, but other interfaces, such as displaying a tree of options for a hierarchical vocabulary, are possible.
578580
Similarly, existing scripts do relatively simple things for displaying a term - showing the term's name in the appropriate language and providing a link to an external URL with more information, but more sophisticated displays are possible.
579581

580-
Scripts supporting use of vocabularies from services supporting the SKOMOS protocol (see https://skosmos.org) and retrieving ORCIDs (from https://orcid.org) are available https://github.com/gdcc/dataverse-external-vocab-support. (Custom scripts can also be used and community members are encouraged to share new scripts through the dataverse-external-vocab-support repository.)
582+
Scripts supporting use of vocabularies from services supporting the SKOMOS protocol (see https://skosmos.org), retrieving ORCIDs (from https://orcid.org), and using ROR (https://ror.org/) are available https://github.com/gdcc/dataverse-external-vocab-support. (Custom scripts can also be used and community members are encouraged to share new scripts through the dataverse-external-vocab-support repository.)
581583

582-
Configuration involves specifying which fields are to be mapped, whether free-text entries are allowed, which vocabulary(ies) should be used, what languages those vocabulary(ies) are available in, and several service protocol and service instance specific parameters.
584+
Configuration involves specifying which fields are to be mapped, whether free-text entries are allowed, which vocabulary(ies) should be used, what languages those vocabulary(ies) are available in, and several service protocol and service instance specific parameters, including the ability to send HTTP headers on calls to the service.
583585
These are all defined in the :ref:`:CVocConf <:CVocConf>` setting as a JSON array. Details about the required elements as well as example JSON arrays are available at https://github.com/gdcc/dataverse-external-vocab-support, along with an example metadata block that can be used for testing.
584586
The scripts required can be hosted locally or retrieved dynamically from https://gdcc.github.io/ (similar to how dataverse-previewers work).
585587

‎doc/sphinx-guides/source/installation/config.rst‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4478,9 +4478,13 @@ A boolean setting that, if true, will send an email and notification to users wh
44784478
:CVocConf
44794479
+++++++++
44804480

4481-
A JSON-structured setting that configures Dataverse to associate specific metadatablock fields with external vocabulary services and specific vocabularies/sub-vocabularies managed by that service. More information about this capability is available at :doc:`/admin/metadatacustomization`.
4481+
The ``:CVocConf`` database setting is used to allow metadatablock fields to look up values in external vocabulary services. For example, you could configure the "Author Affiliation" field to look up organizations in the `Research Organization Registry (ROR) <https://ror.org>`_. For a high-level description of this feature, see :ref:`using-external-vocabulary-services` in the Admin Guide.
44824482

4483-
Scripts that implement this association for specific service protocols are maintained at https://github.com/gdcc/dataverse-external-vocab-support. That repository also includes a json-schema for validating the structure required by this setting along with an example metadatablock and sample :CVocConf setting values associating entries in the example block with ORCID and SKOSMOS based services.
4483+
The expected format for the ``:CVocConf`` database setting is JSON but the details are not documented here. Instead, please refer to `docs/readme.md <https://github.com/gdcc/dataverse-external-vocab-support/blob/main/docs/readme.md>`_ in the https://github.com/gdcc/dataverse-external-vocab-support repo.
4484+
4485+
That repository also includes scripts that implement the lookup for specific service protocols, a JSON Schema for validating the structure required by this setting, and an example metadatablock with a sample ``:CVocConf`` config that associates fields in the example block with ORCID and SKOSMOS based services.
4486+
4487+
The commands below should give you an idea of how to load the configuration, but you'll want to study the examples and make decisions about which configuration to use:
44844488

44854489
``wget https://gdcc.github.io/dataverse-external-vocab-support/examples/config/cvoc-conf.json``
44864490

‎src/main/java/edu/harvard/iq/dataverse/DatasetFieldServiceBean.java‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -505,7 +505,14 @@ public void process(HttpResponse response, HttpContext context) throws HttpExcep
505505
HttpGet httpGet = new HttpGet(retrievalUri);
506506
//application/json+ld is for backward compatibility
507507
httpGet.addHeader("Accept", "application/ld+json, application/json+ld, application/json");
508-
508+
//Adding others custom HTTP request headers if exists
509+
final JsonObject headers = cvocEntry.getJsonObject("headers");
510+
if (headers != null) {
511+
final Set<String> headerKeys = headers.keySet();
512+
for (final String hKey: headerKeys) {
513+
httpGet.addHeader(hKey, headers.getString(hKey));
514+
}
515+
}
509516
HttpResponse response = httpClient.execute(httpGet);
510517
String data = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
511518
int statusCode = response.getStatusLine().getStatusCode();

‎src/main/webapp/datasetFieldForEditFragment.xhtml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@
4242
</c:if>
4343
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvoc.getString('languages'))}"/>
4444
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvoc.getString('cvoc-url','')}"/>
45+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvoc.containsKey('headers') ? cvoc.get('headers').toString() : '{}'}"/>
4546
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvoc.getString('protocol','')}"/>
4647
<f:passThroughAttribute name="data-cvoc-filter" value="#{cvoc.getString('term-parent-uri','')}"/>
4748
<f:passThroughAttribute name="data-cvoc-vocabs" value="#{cvoc.get('vocabs').toString()}"/>

‎src/main/webapp/metadataFragment.xhtml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@
8888
escape="#{dsf.datasetFieldType.isEscapeOutputText()}">
8989
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(dsf.datasetFieldType.id).getString('languages'))}" />
9090
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(dsf.datasetFieldType.id).getString('cvoc-url','')}" />
91+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(dsf.datasetFieldType.id).containsKey('headers') ? cvocConf.get(dsf.datasetFieldType.id).get('headers').toString() : '{}'}"/>
9192
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(dsf.datasetFieldType.id).getString('protocol','')}" />
9293
<f:passThroughAttribute name="data-cvoc-managedfields" value="#{cvocConf.get(dsf.datasetFieldType.id).get('managedfields').toString()}" />
9394
</h:outputText>
@@ -100,6 +101,7 @@
100101
escape="#{dsf.datasetFieldType.isEscapeOutputText()}">
101102
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(dsf.datasetFieldType.id).getString('languages'))}" />
102103
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(dsf.datasetFieldType.id).getString('cvoc-url','')}" />
104+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(dsf.datasetFieldType.id).containsKey('headers') ? cvocConf.get(dsf.datasetFieldType.id).get('headers').toString() : '{}'}"/>
103105
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(dsf.datasetFieldType.id).getString('protocol','')}" />
104106
<f:passThroughAttribute name="data-cvoc-managedfields" value="#{cvocConf.get(dsf.datasetFieldType.id).get('managedfields').toString()}" />
105107
</h:outputText>
@@ -146,6 +148,7 @@
146148
rendered="${cvocOnCvPart and cvPart.key.datasetFieldType.name.equals(cvocConf.get(cvPart.key.datasetFieldType.id).getString('term-uri-field'))}">
147149
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(cvPart.key.datasetFieldType.id).getString('languages'))}" />
148150
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(cvPart.key.datasetFieldType.id).getString('cvoc-url','')}" />
151+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(cvPart.key.datasetFieldType.id).containsKey('headers') ? cvocConf.get(cvPart.key.datasetFieldType.id).get('headers').toString() : '{}'}"/>
149152
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(cvPart.key.datasetFieldType.id).getString('protocol','')}" />
150153
<!-- unlikely to be used in this case -->
151154
<f:passThroughAttribute name="data-cvoc-managedfields" value="#{cvocConf.get(cvPart.key.datasetFieldType.id).get('managedfields').toString()}" />
@@ -156,6 +159,7 @@
156159
rendered="${cvocOnDsf and cvPart.key.datasetFieldType.name.equals(cvocConf.get(dsf.datasetFieldType.id).getString('term-uri-field'))}">
157160
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(dsf.datasetFieldType.id).getString('languages'))}" />
158161
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(dsf.datasetFieldType.id).getString('cvoc-url','')}" />
162+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(dsf.datasetFieldType.id).containsKey('headers') ? cvocConf.get(dsf.datasetFieldType.id).get('headers').toString() : '{}'}"/>
159163
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(dsf.datasetFieldType.id).getString('protocol','')}" />
160164
<f:passThroughAttribute name="data-cvoc-managedfields" value="#{cvocConf.get(dsf.datasetFieldType.id).get('managedfields').toString()}" />
161165
</h:outputText>

‎src/main/webapp/search-include-fragment.xhtml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -213,6 +213,7 @@
213213
<c:if test="#{!cvocConf.isEmpty()}">
214214
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(facetCategory.datasetFieldTypeId).getString('languages'))}" />
215215
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(facetCategory.datasetFieldTypeId).getString('cvoc-url','')}" />
216+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(facetCategory.datasetFieldTypeId).containsKey('headers') ? cvocConf.get(facetCategory.datasetFieldTypeId).get('headers').toString() : '{}'}"/>
216217
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(facetCategory.datasetFieldTypeId).getString('protocol','')}" />
217218
</c:if>
218219
</h:outputText>
@@ -234,6 +235,7 @@
234235
<c:if test="#{!cvocConf.isEmpty()}">
235236
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(facetCategory.datasetFieldTypeId).getString('languages'))}" />
236237
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(facetCategory.datasetFieldTypeId).getString('cvoc-url','')}" />
238+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(facetCategory.datasetFieldTypeId).containsKey('headers') ? cvocConf.get(facetCategory.datasetFieldTypeId).get('headers').toString() : '{}'}"/>
237239
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(facetCategory.datasetFieldTypeId).getString('protocol','')}" />
238240
</c:if>
239241
</h:outputText>
@@ -384,6 +386,7 @@
384386
<h:outputText value="#{friendlyNames.get(1)}">
385387
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(SearchIncludeFragment.getFieldTypeId(friendlyNames.get(0))).getString('languages'))}" />
386388
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(SearchIncludeFragment.getFieldTypeId(friendlyNames.get(0))).getString('cvoc-url','')}" />
389+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(SearchIncludeFragment.getFieldTypeId(friendlyNames.get(0))).containsKey('headers') ? cvocConf.get(SearchIncludeFragment.getFieldTypeId(friendlyNames.get(0))).get('headers').toString() : '{}'}"/>
387390
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(SearchIncludeFragment.getFieldTypeId(friendlyNames.get(0))).getString('protocol','')}" />
388391
</h:outputText>
389392
<span class="glyphicon glyphicon-remove"></span>
@@ -621,6 +624,7 @@
621624
<c:if test="#{!cvocConf.isEmpty()}">
622625
<f:passThroughAttribute name="lang" value="#{DatasetPage.getFieldLanguage(cvocConf.get(datasetFieldServiceBean.findByName(highlight.solrField.nameSearchable).id).getString('languages'))}" />
623626
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvocConf.get(datasetFieldServiceBean.findByName(highlight.solrField.nameSearchable).id).getString('cvoc-url','')}" />
627+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvocConf.get(datasetFieldServiceBean.findByName(highlight.solrField.nameSearchable).id).containsKey('headers') ? cvocConf.get(datasetFieldServiceBean.findByName(highlight.solrField.nameSearchable).id).get('headers').toString() : '{}'}"/>
624628
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvocConf.get(datasetFieldServiceBean.findByName(highlight.solrField.nameSearchable).id).getString('protocol','')}" />
625629
</c:if>
626630
</h:outputText>

‎src/main/webapp/search/advanced.xhtml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,7 @@
121121
<p:inputText id="searchValue2" styleClass="form-control" value="#{item.searchValue}" rendered="#{empty item.controlledVocabularyValues}">
122122
<f:passThroughAttribute name="lang" value="#{AdvancedSearchPage.getFieldLanguage(cvoc.getString('languages'))}"/>
123123
<f:passThroughAttribute name="data-cvoc-service-url" value="#{cvoc.getString('cvoc-url','')}"/>
124+
<f:passThroughAttribute name="data-cvoc-headers" value="#{cvoc.containsKey('headers') ? cvoc.get('headers').toString() : '{}'}"/>
124125
<f:passThroughAttribute name="data-cvoc-protocol" value="#{cvoc.getString('protocol','')}"/>
125126
<f:passThroughAttribute name="data-cvoc-filter" value="#{cvoc.getString('term-parent-uri','')}"/>
126127
<f:passThroughAttribute name="data-cvoc-vocabs" value="#{cvoc.get('vocabs').toString()}"/>

‎src/test/java/edu/harvard/iq/dataverse/api/SearchIT.java‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -862,7 +862,7 @@ public void testNestedSubtree() {
862862
Response publishDataset = UtilIT.publishDatasetViaNativeApi(datasetPid, "major", apiToken);
863863
publishDataset.then().assertThat()
864864
.statusCode(OK.getStatusCode());
865-
865+
UtilIT.sleepForReindex(datasetPid, apiToken, 5);
866866
Response searchPublishedSubtreeWDS = UtilIT.search(searchPart, apiToken, "&subtree="+dataverseAlias);
867867
searchPublishedSubtreeWDS.prettyPrint();
868868
searchPublishedSubtreeWDS.then().assertThat()

0 commit comments

Comments
 (0)
⚡