Describe the bug
If index is created with "index.derived_source.enabled": true and excludes/includes is specified in the mappings/search request, the includes and excludes are not honored. We need to filter out the excludes and always need the includes in the code path with derived source feature.
Related component
No response
To Reproduce
curl -X PUT "http://localhost:9200/test-core-excludes" -H "Content-Type: application/json" -d '{
"settings": {
"index": {
"derived_source.enabled": true
}
},
"mappings": {
"_source": {
"excludes": ["age"]
},
"properties": {
"name": { "type": "keyword" },
"age": { "type": "integer" },
"city": { "type": "keyword" }
}
}
}'
curl -X PUT "http://localhost:9200/test-core-excludes/_doc/1" -H "Content-Type: application/json" -d '{
"name": "test",
"age": 25,
"city": "seattle"
}'
curl -X GET "http://localhost:9200/test-core-excludes/_doc/1?pretty"
{
"_index" : "test-core-excludes",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"name" : "test",
"city" : "seattle",
"age" : 25
}
}
The response still includes the age field even though it is excluded.
Expected behavior
curl -X GET "http://localhost:9200/test-core-excludes/_doc/1?pretty"
{
"_index" : "test-core-excludes",
"_id" : "1",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"name" : "test",
"city" : "seattle"
}
}
Age should be excluded in the response.
Additional Details
Plugins
opensearch-project/k-NN#3260 - PR enables KNN based feature with core is also impacted with this bug.
Describe the bug
If index is created with "index.derived_source.enabled": true and excludes/includes is specified in the mappings/search request, the includes and excludes are not honored. We need to filter out the excludes and always need the includes in the code path with derived source feature.
Related component
No response
To Reproduce
The response still includes the age field even though it is excluded.
Expected behavior
curl -X GET "http://localhost:9200/test-core-excludes/_doc/1?pretty"
Age should be excluded in the response.
Additional Details
Plugins
opensearch-project/k-NN#3260 - PR enables KNN based feature with core is also impacted with this bug.