Skip to content

feat(instrumentation-openai): Migrate OpenAI instrumentation to OTel 1.40 semantic conventions #909

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dvirski merged 12 commits into from
Mar 30, 2026
Merged

feat(instrumentation-openai): Migrate OpenAI instrumentation to OTel 1.40 semantic conventions #909

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
9b7f332
feat(instrumentation-openai): Migrate OpenAI instrumentation to OTel …
dvirski Mar 29, 2026
eeb7d5e
Fix prettier
dvirski Mar 29, 2026
1a62402
update lock file for traceloop-sdk
dvirski Mar 29, 2026
6fcadd6
CR Changes
dvirski Mar 29, 2026
34529f2
CR Changes
dvirski Mar 29, 2026
511c570
Prettier fixes
dvirski Mar 29, 2026
0d61000
Reverted back openai sdk package back to 4.38.3, since polly can't in…
dvirski Mar 30, 2026
3b6eff3
Added tests for open ai builder and block mapper functions
dvirski Mar 30, 2026
b7482fb
fixed style errors
dvirski Mar 30, 2026
8ea6656
semconv review changes
dvirski Mar 30, 2026
7319818
fixed styles
dvirski Mar 30, 2026
3d23f46
CR Fix
dvirski Mar 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions packages/instrumentation-openai/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,9 @@
"@opentelemetry/api": "^1.9.0",
"@opentelemetry/core": "^2.0.1",
"@opentelemetry/instrumentation": "^0.203.0",
"@opentelemetry/semantic-conventions": "^1.38.0",
"@opentelemetry/semantic-conventions": "^1.40.0",
"@traceloop/ai-semantic-conventions": "workspace:*",
"@traceloop/instrumentation-utils": "workspace:*",
"js-tiktoken": "^1.0.20",
"tslib": "^2.8.1"
},
Expand All @@ -57,7 +58,7 @@
"@types/node-fetch": "^2.6.13",
"mocha": "^11.7.1",
"node-fetch": "^2.7.0",
"openai": "5.12.2",
"openai": "^6.32.0",
"ts-mocha": "^11.1.0"
},
"homepage": "https://github.com/traceloop/openllmetry-js/tree/main/packages/instrumentation-openai",
Expand Down
102 changes: 59 additions & 43 deletions packages/instrumentation-openai/src/image-wrappers.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,14 @@
import { trace, Span, SpanKind, Attributes } from "@opentelemetry/api";
import { SpanAttributes } from "@traceloop/ai-semantic-conventions";
import {
ATTR_GEN_AI_COMPLETION,
ATTR_GEN_AI_PROMPT,
ATTR_GEN_AI_REQUEST_MODEL,
ATTR_GEN_AI_SYSTEM,
ATTR_GEN_AI_USAGE_COMPLETION_TOKENS,
ATTR_GEN_AI_USAGE_PROMPT_TOKENS,
ATTR_GEN_AI_PROVIDER_NAME,
ATTR_GEN_AI_INPUT_MESSAGES,
ATTR_GEN_AI_OUTPUT_MESSAGES,
ATTR_GEN_AI_USAGE_INPUT_TOKENS,
ATTR_GEN_AI_USAGE_OUTPUT_TOKENS,
ATTR_GEN_AI_OPERATION_NAME,
GEN_AI_PROVIDER_NAME_VALUE_OPENAI,
} from "@opentelemetry/semantic-conventions/incubating";
import type { ImageUploadCallback } from "./types";
import type {
Expand Down Expand Up @@ -169,8 +171,9 @@ export function setImageGenerationRequestAttributes(
}

if (params.prompt) {
attributes[`${ATTR_GEN_AI_PROMPT}.0.content`] = params.prompt;
attributes[`${ATTR_GEN_AI_PROMPT}.0.role`] = "user";
attributes[ATTR_GEN_AI_INPUT_MESSAGES] = JSON.stringify([
{ role: "user", parts: [{ type: "text", content: params.prompt }] },
Copy link
Copy Markdown
Contributor

@nina-kollman nina-kollman Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it would be cleaner to have a util function the creates this json and it will be used all over the place.
WDYT?

Copy link
Copy Markdown
Contributor Author

@dvirski dvirski Mar 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not worth extracting further — it's only 2 places, and they're building arrays they may extend with image parts. A function that returns one message object would save almost nothing and add indirection.

]);
}

Object.entries(attributes).forEach(([key, value]) => {
Expand Down Expand Up @@ -200,8 +203,9 @@ export async function setImageEditRequestAttributes(
}

if (params.prompt) {
attributes[`${ATTR_GEN_AI_PROMPT}.0.content`] = params.prompt;
attributes[`${ATTR_GEN_AI_PROMPT}.0.role`] = "user";
attributes[ATTR_GEN_AI_INPUT_MESSAGES] = JSON.stringify([
{ role: "user", parts: [{ type: "text", content: params.prompt }] },
]);
}

// Process input image if upload callback is available
Expand All @@ -223,10 +227,23 @@ export async function setImageEditRequestAttributes(
);

if (imageUrl) {
attributes[`${ATTR_GEN_AI_PROMPT}.1.content`] = JSON.stringify([
{ type: "image_url", image_url: { url: imageUrl } },
]);
attributes[`${ATTR_GEN_AI_PROMPT}.1.role`] = "user";
// Add the image as a second input message
const existingMessages = attributes[ATTR_GEN_AI_INPUT_MESSAGES];
if (existingMessages) {
const parsed = JSON.parse(existingMessages as string);
parsed.push({
role: "user",
parts: [{ type: "uri", modality: "image", uri: imageUrl }],
});
attributes[ATTR_GEN_AI_INPUT_MESSAGES] = JSON.stringify(parsed);
} else {
attributes[ATTR_GEN_AI_INPUT_MESSAGES] = JSON.stringify([
{
role: "user",
parts: [{ type: "uri", modality: "image", uri: imageUrl }],
},
]);
}
}
}

Expand Down Expand Up @@ -275,10 +292,12 @@ export async function setImageVariationRequestAttributes(
);

if (imageUrl) {
attributes[`${ATTR_GEN_AI_PROMPT}.0.content`] = JSON.stringify([
{ type: "image_url", image_url: { url: imageUrl } },
attributes[ATTR_GEN_AI_INPUT_MESSAGES] = JSON.stringify([
{
role: "user",
parts: [{ type: "uri", modality: "image", uri: imageUrl }],
},
]);
attributes[`${ATTR_GEN_AI_PROMPT}.0.role`] = "user";
}
}

Expand All @@ -303,7 +322,7 @@ export async function setImageGenerationResponseAttributes(
params,
response.data.length,
);
attributes[ATTR_GEN_AI_USAGE_COMPLETION_TOKENS] = completionTokens;
attributes[ATTR_GEN_AI_USAGE_OUTPUT_TOKENS] = completionTokens;

// Calculate prompt tokens if enrichTokens is enabled
if (instrumentationConfig?.enrichTokens) {
Expand All @@ -319,38 +338,34 @@ export async function setImageGenerationResponseAttributes(
}

if (estimatedPromptTokens > 0) {
attributes[ATTR_GEN_AI_USAGE_PROMPT_TOKENS] = estimatedPromptTokens;
attributes[ATTR_GEN_AI_USAGE_INPUT_TOKENS] = estimatedPromptTokens;
}

attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] =
attributes[SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS] =
estimatedPromptTokens + completionTokens;
} catch {
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = completionTokens;
attributes[SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS] = completionTokens;
}
} else {
attributes[SpanAttributes.LLM_USAGE_TOTAL_TOKENS] = completionTokens;
attributes[SpanAttributes.GEN_AI_USAGE_TOTAL_TOKENS] = completionTokens;
}
}

if (response.data && response.data.length > 0) {
const firstImage = response.data[0];
let imageOutputUrl: string | undefined;

if (firstImage.b64_json && uploadCallback) {
try {
const traceId = span.spanContext().traceId;
const spanId = span.spanContext().spanId;

const imageUrl = await uploadCallback(
imageOutputUrl = await uploadCallback(
traceId,
spanId,
"generated_image.png",
firstImage.b64_json,
);

attributes[`${ATTR_GEN_AI_COMPLETION}.0.content`] = JSON.stringify([
{ type: "image_url", image_url: { url: imageUrl } },
]);
attributes[`${ATTR_GEN_AI_COMPLETION}.0.role`] = "assistant";
} catch (error) {
console.error("Failed to upload generated image:", error);
}
Expand All @@ -364,29 +379,27 @@ export async function setImageGenerationResponseAttributes(
const buffer = Buffer.from(arrayBuffer);
const base64Data = buffer.toString("base64");

const uploadedUrl = await uploadCallback(
imageOutputUrl = await uploadCallback(
traceId,
spanId,
"generated_image.png",
base64Data,
);

attributes[`${ATTR_GEN_AI_COMPLETION}.0.content`] = JSON.stringify([
{ type: "image_url", image_url: { url: uploadedUrl } },
]);
attributes[`${ATTR_GEN_AI_COMPLETION}.0.role`] = "assistant";
} catch (error) {
console.error("Failed to fetch and upload generated image:", error);
attributes[`${ATTR_GEN_AI_COMPLETION}.0.content`] = JSON.stringify([
{ type: "image_url", image_url: { url: firstImage.url } },
]);
attributes[`${ATTR_GEN_AI_COMPLETION}.0.role`] = "assistant";
imageOutputUrl = firstImage.url;
}
} else if (firstImage.url) {
attributes[`${ATTR_GEN_AI_COMPLETION}.0.content`] = JSON.stringify([
{ type: "image_url", image_url: { url: firstImage.url } },
imageOutputUrl = firstImage.url;
}

if (imageOutputUrl) {
attributes[ATTR_GEN_AI_OUTPUT_MESSAGES] = JSON.stringify([
{
role: "assistant",
parts: [{ type: "uri", modality: "image", uri: imageOutputUrl }],
},
]);
attributes[`${ATTR_GEN_AI_COMPLETION}.0.role`] = "assistant";
}

if (firstImage.revised_prompt) {
Expand All @@ -413,7 +426,8 @@ export function wrapImageGeneration(
const span = tracer.startSpan("openai.images.generate", {
kind: SpanKind.CLIENT,
attributes: {
[ATTR_GEN_AI_SYSTEM]: "OpenAI",
[ATTR_GEN_AI_PROVIDER_NAME]: GEN_AI_PROVIDER_NAME_VALUE_OPENAI,
[ATTR_GEN_AI_OPERATION_NAME]: "image_generation",
"gen_ai.request.type": "image_generation",
},
});
Expand Down Expand Up @@ -472,7 +486,8 @@ export function wrapImageEdit(
const span = tracer.startSpan("openai.images.edit", {
kind: SpanKind.CLIENT,
attributes: {
[ATTR_GEN_AI_SYSTEM]: "OpenAI",
[ATTR_GEN_AI_PROVIDER_NAME]: GEN_AI_PROVIDER_NAME_VALUE_OPENAI,
[ATTR_GEN_AI_OPERATION_NAME]: "image_edit",
"gen_ai.request.type": "image_edit",
},
});
Expand Down Expand Up @@ -539,7 +554,8 @@ export function wrapImageVariation(
const span = tracer.startSpan("openai.images.createVariation", {
kind: SpanKind.CLIENT,
attributes: {
[ATTR_GEN_AI_SYSTEM]: "OpenAI",
[ATTR_GEN_AI_PROVIDER_NAME]: GEN_AI_PROVIDER_NAME_VALUE_OPENAI,
[ATTR_GEN_AI_OPERATION_NAME]: "image_variation",
"gen_ai.request.type": "image_variation",
},
});
Expand Down
Loading
Loading
⚡