Decouple exporters from processors

[thompson-tomo]

Looking at the schema definition one of things which has stood out is the the way in which extension points can be nested within extension points. This works ok when it is being centrally managed however it adds complexity when you start to think about the amount of integrations we have.

What I would like to propose is decoupling exporters from being children of processors to being sibling. This change would result in the following:

{
    "$id": "https://opentelemetry.io/otelconfig/logger_provider.json",
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "additionalProperties": false,
    "properties": {
        "processors": {
            "type": "array",
            "minItems": 1,
            "items": {
                "$ref": "#/$defs/LogRecordProcessor"
            }
        }
....
    },
    "required": [
        "processors"
    ],
    "$defs": {
        "SimpleLogRecordProcessor": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
....
                "exporter": {
                    "$ref": "#/$defs/LogRecordExporter"
                }
            },
            "required": [
                "exporter"
            ]
        },
        "BatchLogRecordProcessor": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
....
                "exporter": {
                    "$ref": "#/$defs/LogRecordExporter"
                }
            },
            "required": [
                "exporter"
            ]
        },
        "LogRecordExporter": {
            "type": "object",
            "additionalProperties": {
                "type": ["object", "null"]
            },
            "minProperties": 1,
            "maxProperties": 1,
            "properties": {
                "otlp_http": {
                    "$ref": "common.json#/$defs/OtlpHttpExporter"
                },
                "otlp_grpc": {
                    "$ref": "common.json#/$defs/OtlpGrpcExporter"
                },
                "otlp_file/development": {
                    "$ref": "common.json#/$defs/ExperimentalOtlpFileExporter"
                },
                "console": {
                    "$ref": "common.json#/$defs/ConsoleExporter"
                }
            }
        },
        "LogRecordProcessor": {
            "type": "object",
            "additionalProperties": {
                "type": ["object", "null"]
            },
            "minProperties": 1,
            "maxProperties": 1,
            "properties": {
                "batch": {
                    "$ref": "#/$defs/BatchLogRecordProcessor"
                },
                "simple": {
                    "$ref": "#/$defs/SimpleLogRecordProcessor"
                }
            }
        }
   }
}

Becoming

{
    "$id": "https://opentelemetry.io/otelconfig/logger_provider.json",
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "type": "object",
    "additionalProperties": false,
    "properties": {
        "processors": {
            "type": "array",
            "minItems": 1,
            "items": {
                "$ref": "#/$defs/LogRecordProcessor"
            }
        },
        "exporters": {
            "type": "array",
            "minItems": 1,
            "items": {
                "$ref": "#/$defs/LogRecordExporter"
            }
....
    },
    "required": [
        "processors", ""
    ],
    "$defs": {
        "SimpleLogRecordProcessor": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
....
            }
        },
        "BatchLogRecordProcessor": {
            "type": "object",
            "additionalProperties": false,
            "properties": {
....
            }
        },
        "LogRecordExporter": {
            "type": "object",
            "additionalProperties": {
                "type": ["object", "null"]
            },
            "minProperties": 1,
            "maxProperties": 1,
            "properties": {
                "otlp_http": {
                    "$ref": "common.json#/$defs/OtlpHttpExporter"
                },
                "otlp_grpc": {
                    "$ref": "common.json#/$defs/OtlpGrpcExporter"
                },
                "otlp_file/development": {
                    "$ref": "common.json#/$defs/ExperimentalOtlpFileExporter"
                },
                "console": {
                    "$ref": "common.json#/$defs/ConsoleExporter"
                }
            }
        },
        "LogRecordProcessor": {
            "type": "object",
            "additionalProperties": {
                "type": ["object", "null"]
            },
            "minProperties": 1,
            "maxProperties": 1,
            "properties": {
                "batch": {
                    "$ref": "#/$defs/BatchLogRecordProcessor"
                },
                "simple": {
                    "$ref": "#/$defs/SimpleLogRecordProcessor"
                }
            }
        }
   }
}

With this change there is no requirement on processor to define an exporter field which is a big plus if we get a position of collecting schemas for all the different otel components. It would also better enable Scoping of config exposed to modules to only be what is needed.

This approach also ties into the collector & sdk's which are already use to having processors & exporters.

[jack-berg]

This issue goes back to the very earliest days of the project where we were developing the schema in a repo outside of the opentelemetry poject in preparation for our otep: https://github.com/MrAlias/otel-schema

Here are some old issues / PRs where this was discussed:

• Define processors/exporters relationship #8
• Should exporter configuration be nested or at top level? MrAlias/otel-schema#12
• Add tracer_provider exporters and span_processors MrAlias/otel-schema#17
• Add span processors configuration MrAlias/otel-schema#16
• Configuration proposal oteps#225 (comment)

At this stage of the project, we're coming up on 3 years in, and we've cut 2 release candidates. Stabilization is the top priority right now and we're looking for bug fixes and gross oversights. A change like this is a significant redesign that would be a breaking change across several parts of the schema and open many followup questions.

I'm going to close this as "not planned".