Wrap Entries Processor - Behavior Discussion

[nishantKadivar]

Wrap Entries Processor

A processor that wraps each element of a list into an object using a configured key name. This is particularly useful for transforming primitive arrays (strings, numbers, booleans) into arrays of objects with a uniform structure, enabling downstream processors like add_entries and delete_entries with iterate_on capability, which require List<Map<String, Object>>.

Overview

The wrap_entries processor transforms:

• Primitive arrays (strings, numbers, booleans) → Arrays of objects
• Non-primitive arrays (maps, lists) → Nested wrapped objects
  This enables consistent data structure handling across the pipeline.

---

Basic Usage

Create the following pipeline.yaml:

pipeline:
  source:
    file:
      path: "/full/path/to/input.json"
      record_type: "event"
      format: "json"
  processor:
    - wrap_entries:
        source: "/names"
        key: "name"
  sink:
    - stdout:

Create the following file named input.json:

{"names": ["alice", "bob", "charlie"]}

When run, the processor will output:

{"names": [{"name": "alice"}, {"name": "bob"}, {"name": "charlie"}]}

---

Configuration

Option	Required	Type	Default	Description
source	Yes	String	—	Key of the array to transform (JSON Pointer format, e.g., /names or /users/0/items)
target	No	String	source	Key to write the result to. If not specified, the result overwrites the source array in-place
key	Yes	String	—	Key name to wrap each element with in the resulting object
exclude_null_empty_values	No	Boolean	false	Filter out null and empty string ("") elements before wrapping. If all elements are filtered out, the result is an empty array []
append_if_target_exists	No	Boolean	false	When true, appends wrapped elements to an existing target array instead of overwriting it
wrap_entries_when	No	String	null	Conditional expression to evaluate at the root event level. The processor only executes if the condition is true
tags_on_failure	No	List	null	Tags to add to event metadata when the processor fails to transform the array

---

Example 1: Non-Primitive Array (Array of Maps) ⚠️ DISCUSSION POINT

Configuration:

processor:
  - wrap_entries:
      source: "/users"
      key: "user"

Input:

{
  "users": [
    {"id": 1, "name": "alice"},
    {"id": 2, "name": "bob"}
  ]
}

Current Output (WRAPS NON-PRIMITIVE ELEMENTS):

{
  "users": [
    {"user": {"id": 1, "name": "alice"}},
    {"user": {"id": 2, "name": "bob"}}
  ]
}

⚠️ UNDER DISCUSSION: Should the processor wrap map elements?

• Option A: Yes, wrap everything (current behavior)
• Option B: No, leave maps untouched
• Option C: No, raise an error

---

Example 2: Non-Primitive Array (Array of Arrays/Lists) ⚠️ DISCUSSION POINT

Configuration:

processor:
  - wrap_entries:
      source: "/items"
      key: "item"

Input:

{
  "items": [
    ["alpha1", "beta1"],
    ["alpha2", "beta2"]
  ]
}

Current Output (WRAPS NON-PRIMITIVE ELEMENTS):

{
  "items": [
    {"item": ["alpha1", "beta1"]},
    {"item": ["alpha2", "beta2"]}
  ]
}

⚠️ UNDER DISCUSSION: Should the processor wrap list/array elements?

• Option A: Yes, wrap everything (current behavior)
• Option B: No, leave lists untouched
• Option C: No, raise an error

---

Behavior Discussion

Current Status ⚠️

The wrap_entries processor currently wraps all array elements, including non-primitives (maps and lists). This behavior is functional but requires community discussion before the 2.16 release.

Questions for Decision

1. Should non-primitive elements be wrapped?

   • Example 3 shows array of maps being wrapped as {"user": {...}}
   • Example 4 shows array of arrays being wrapped as {"item": [...]}
   • Should these behaviors be supported or restricted?

2. What is the primary use case?

   • Enable add_entries/delete_entries with iterate_on (requires List<Map<String, Object>>)
   • General-purpose array transformation
   • Both?

3. Should this behavior be released in 2.16?

   • Should the decision be finalized before release?
   • Should documentation reflect this behavior or potential changes?

[dlvenable]

I think for all of these behaviors we want to give options. One question is what is the default? Based on my experience with customer pipelines we want defaults that promote trying to produce consistent output.