Log stabilization current PRs overview

[xrmx]

Plan

PR/issue	impact	alternatives	notes
#4676	Removal of sdk LogData (no substitute) and LogRecord (now ReadableLogRecord and ReadWriteLogRecord). Change of typing in LogRecordProcessor.on_emit, in LogExporter.export. This will break openllmetry tests if they test against latest		MERGED
#4654	deprecations of _events, typechecking issues for users		MERGED
#4647	Renames a bunch of sdk and exporter Log related classes, some of the sdk classes are touched in #4676 . Most if not all the old classes names are kept around with a deprecation notice		MERGED
open-telemetry/opentelemetry-python-contrib#3628	none		MERGED, t, nlu
open-telemetry/opentelemetry-python-contrib#3626	none		MERGED t, nlu
open-telemetry/opentelemetry-python-contrib#3625	none		MERGED t, nlu
open-telemetry/opentelemetry-python-contrib#3624	none		MERGED, t, nlu
#4319	this contains a suggestion to change the signature of Logger.emit to take separated params instead of a a LogRecord, this can break all the genai instrumentations	we can do an implementation having both LogRecord and deprecate it later? e.g. #4737	MERGED
#4330	When moving the loggingHandler from sdk to separate instrumentation, people without the logging instrumentation installed will lose the functionality		MERGED open-telemetry/opentelemetry-python-contrib#4263, #4919, open-telemetry/opentelemetry-python-contrib#4210
open-telemetry/opentelemetry-python-contrib#4112	Removal of out of spec attributes from opentelemetry-instrumentation-logging when log formatting is not configured		MERGED

nlu = introduce a new direct users for _logs api
t = can be used as testbed for out of tree users

Feedback

issue	impact	notes	fixed
#4836	warnings when using test utils	from #4654	yes
#4783	confusion	handled by #4676,	yes
#4838	breakage	we can probably re-export LogData as deprecated inherited from ReadableLogRecord	no

[aabmass]

I don't think any of the contrib issues are strictly needed for stabilization. We should address #4330 IMO.

FYI we have a label "log-breaking-changes" (issues) to track as well.

[xrmx]

I don't think any of the contrib issues are strictly needed for stabilization. We should address #4330 IMO.

Added the LoggingHandler move to the table. I've listed the contrib changes because they are users of some of the code we are touching and so helpful to reason about the impact.

FYI we have a label "log-breaking-changes" (issues) to track as well.

The other labeled issue is fixed by a PR listed in the table (#4647)

[xrmx]

So here's a try at serializing all these PRs:

• find some downstream users that may be interested in checking their stuff against OTel git. Do we have big downstream users of Events? openllmetry, we should ask them to test the events deprecation branch then :) Filed 🐛 Bug Report: Switch to log events on recent OpenTelemetry sdks traceloop/openllmetry#3451
• decide if we want to do the Logger.emit interface change Logs API: not possible to use without SDK #4319, if so start with a version that keeps backward compat and find a way to add the deprecations for the older interface
• merge a PR moving from events to logs for a single genai instrumentation and release it, see if any issues arise
• merge events api deprecation Mark events API/SDK as deprecated #4654 and evaluate how annoying it's for the other instrumentations, seek feedback from the downstream people, discuss and adapt accordingly - Dylan suggested to wait for this because users of the instrumentations don't have options. OpenLLMetry is fixed and released, we need to release ours instrumentation:
  • release openai-v2
  • update google-genai
• if first conversion from events to logs worked fine convert the others
• now think really hard on the effects of [logs-sdk] Remove LogData and extend SDK LogRecord to have instrumentation scope #4676, again downstream users can check if something breaks for them
• merge the PR doing the renames [logs-api/sdk] Rename several classes from Log to LogRecord #4647 to double check that the old names are still around.
• adapt to the new names internally and seek for downstream user feedback. This a good time to check the documentation we have around (also in opentelemetry.io) and update that too
• Released 1.39.0 / 1.39.1
• Unnecessary/noncompliant attributes added via logging instrumentation opentelemetry-python-contrib#3687 should be easy to do, sorry for your query if you were using them :)
• Let's move the handler in Consider moving LoggingHandler out of the SDK and into an instrumentation #4330, this is a good chance to review the logging instrumentation code for other issues
• Released 1.40.0

[DylanRussell]

Merging one of the gen AI PRs migrating off events SGTM if we want to play it super safe. I updated all those PRs to require v1.37 of the SDK/API which has your change attaching resource on emit, so they should be good to go..

We may want to wait on the events deprecation stuff for all the gen AI PRs migrating off events to go through, because people using those instrumentations will not be able to do anything about those warnings

[xrmx]

In opentelemetry.io we need to take care of the following references, that also means we don't have documentation for LogRecordLimits:

content/en/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/en/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())
content/pt/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/pt/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())

In opentelemetry-python-contrib, should be taken care of by open-telemetry/opentelemetry-python-contrib#3589:

instrumentation-genai/opentelemetry-instrumentation-google-genai/tests/common/otel_mocker.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-google-genai/tests/common/otel_mocker.py:        self._logs = InMemoryLogExporter()
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/conftest.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-openai-v2/tests/conftest.py:    exporter = InMemoryLogExporter()
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/conftest.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/conftest.py:    exporter = InMemoryLogExporter()
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_chat_completions_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation-genai/opentelemetry-instrumentation-vertexai/tests/test_function_calling_experimental.py:    log_exporter: InMemoryLogExporter,
instrumentation/opentelemetry-instrumentation-botocore/tests/conftest.py:    InMemoryLogExporter,
instrumentation/opentelemetry-instrumentation-botocore/tests/conftest.py:    exporter = InMemoryLogExporter()

[emdneto]

In opentelemetry.io we need to take care of the following references, that also means we don't have documentation for LogRecordLimits:

content/en/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/en/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())
content/pt/docs/languages/python/instrumentation.md:from opentelemetry.sdk._logs.export import BatchLogRecordProcessor, ConsoleLogExporter
content/pt/docs/languages/python/instrumentation.md:processor = BatchLogRecordProcessor(ConsoleLogExporter())

@xrmx I can take care of opentelemetry.io for en/pt localization.

[xrmx]

1.39.0 contains all the changes to logs API and SDK (including events), 1.40.0 should contain the changes for the logging handler.

[xrmx]

Opened open-telemetry/opentelemetry-python-contrib#4112 for removing by default out of spec attribute in log records.