feat: add langfuse plugin for LLM observability#13095
Open
sihyeonn wants to merge 3 commits intoapache:masterfrom
Open
feat: add langfuse plugin for LLM observability#13095sihyeonn wants to merge 3 commits intoapache:masterfrom
sihyeonn wants to merge 3 commits intoapache:masterfrom
Conversation
dosubot
bot
added
size:XXL
Contributor
|
This PR contains three fundamentally different features:
Apache APISIX community practice requires each PR to focus on a single feature. These three features have different review dimensions, testing requirements, and regression risks; please separate them into independent PRs. |
Contributor
|
Hi @sihyeonn, following up on the previous review comments. Please let us know if you have any updates. Thank you. |
sihyeonn
force-pushed
the
feat/langfuse-plugin
branch
from
187a719 to
d3003c0
Compare
Contributor
|
Hi @sihyeonn, my suggestion is to only keep the feature-related code in the current PR, and submit other changes in separate PRs. |
Add a new langfuse plugin that sends AI request traces to Langfuse via its ingestion API using batch-processor-manager pattern. Features: - Auto-detect AI endpoints (configurable patterns) - Generate trace-create and generation-create batch items per request - W3C traceparent header parsing and propagation - Token usage extraction (ai-proxy ctx > nginx vars > response body) - Support for X-Langfuse-Tags and X-Langfuse-Metadata headers - Basic auth with encrypt_fields for secret key protection - Connection keepalive for Langfuse API calls Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Move langfuse_host, langfuse_public_key, langfuse_secret_key, ssl_verify, timeout, detect_ai_requests, and ai_endpoints from per-route schema to metadata_schema (plugin_attr), following the opentelemetry plugin pattern. Per-route schema now only contains include_metadata. Remove encrypt_fields (custom feature, not in OSS). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Remove /v1-prefixed entries since has_suffix matching already covers them via shorter suffixes (e.g. /chat/completions matches both /chat/completions and /v1/chat/completions). Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
sihyeonn
force-pushed
the
feat/langfuse-plugin
branch
from
d3003c0 to
9c4b25a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add a new
langfuseplugin that sends LLM request traces to Langfuse for AI observability. The plugin uses the batch-processor-manager pattern (same as http-logger) to send trace data via Langfuse's ingestion API.Key features:
trace-createandgeneration-createbatch items per requesttraceparentheader parsing and propagation for distributed tracingctx.ai_token_usage(ai-proxy) > nginx vars > response body parsingX-Langfuse-TagsandX-Langfuse-Metadatacustom headersencrypt_fieldsfor secret key protection in etcdWhich issue(s) this PR fixes:
Fixes #
Checklist