Added parsing otel json trace to span from kafka topic

[Mamol27]

Description

Parsing of OpenTelemetry trace in JSON format into Span has been implemented to support trace handling via Kafka.
We had to implement it, as the client requires all data to be sent exclusively through Kafka.
To implement this, a new MessageFormat was added.

Issues Resolved

Resolves #5446

Check List

• New functionality includes testing.
• New functionality has a documentation issue. Please link to it in this PR.
  • New functionality has javadoc added
• Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[kkondaka]

@Mamol27 Thank you for your submission. Wondering if it makes sense to use "JSON" message format (because the input is in json) and then have something like "codec" which would convert JSON to OTEL trace.
Secondly, I am not sure if you need to have JsonOtelTracePojo.java That should be default behavior when using OTelProtoStandardCodec..OTelProtoEncoder.convertToResourceSpans

[KarstenSchnitter]

I think, there is an easier way leveraging the protobuf JsonFormat. This can lead to more code reusage in the mapping of the OTel data. This would be desirable to the project, as it has a lower maintenance effort.

I can only recommend to use the protobuf binary representation to store in Kafka over JSON. There is a much higher throughput in serialization and deserialization of protobuf over JSON.

[KarstenSchnitter]

Thanks for providing this change. This PR contains a lot of whitespace changes. Please check your code formatter configuration.

[Mamol27]

Thanks for providing this change. This PR contains a lot of whitespace changes. Please check your code formatter configuration.

Where is reference to code formatting settings for your project?

[KarstenSchnitter]

Thanks for providing this change. This PR contains a lot of whitespace changes. Please check your code formatter configuration.

Where is reference to code formatting settings for your project?

Have a look at the formatting section in the developer guide.

[Mamol27]

@KarstenSchnitter Hi, could you help me? I ran the IT tests in KafkaBufferIT on my local machine, and they all passed except for two:

read_decrypts_data_from_the_predefined_key and write_and_read_encrypted

Both of those failed with:

Condition with lambda expression in org.opensearch.dataprepper.plugins.kafka.buffer.ReadBufferHelper that uses [Ljava.util.Map$Entry;, [Ljava.util.Map$Entry;org.opensearch.dataprepper.plugins.kafka.buffer.KafkaBuffer was not fulfilled within 30 seconds.
org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.opensearch.dataprepper.plugins.kafka.buffer.ReadBufferHelper that uses [Ljava.util.Map$Entry;, [Ljava.util.Map$Entry;org.opensearch.dataprepper.plugins.kafka.buffer.KafkaBuffer was not fulfilled within 30 seconds.

Could you advise what I should do? This isn’t the same error I see in GitHub Actions.

[KarstenSchnitter]

I have no idea, what might cause this issue. Hopefully one of the other maintainers can help with that.