Skip to content

Added OTel instrumentation and metrics export for async client#3977

Merged
vladvildanov merged 29 commits intomasterfrom
feat/async-observability
Mar 3, 2026
Merged

Added OTel instrumentation and metrics export for async client#3977
vladvildanov merged 29 commits intomasterfrom
feat/async-observability

Conversation

@vladvildanov
Copy link
Copy Markdown
Collaborator

@vladvildanov vladvildanov commented Feb 24, 2026

Description of change

This PR provides recording of metrics for async clients. In terms of public API nothing has been changed so you can use #3954 description or official docs to understand the OTel configuration flow.

We only bring new async recorder functions that is used across async clients to collect the same metrics (where it's possible, CSC and SCH are not available via async clients)

Pull Request check-list

Please make sure to review and check all of these items:

  • Do tests and lints pass with this change?
  • Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
  • Is there an example added to the examples folder (if applicable)?

NOTE: these things are not required to open a PR and can be done
afterwards / while the PR is open.


Note

Medium Risk
Touches async command execution, retry callbacks, and connection lifecycle paths to emit metrics, which could impact error handling and performance under load. Changes are mostly additive/instrumentation but are in high-traffic code paths (client/cluster/connection/pipeline).

Overview
Adds a new async observability layer (redis.asyncio.observability.recorder) and wires it into asyncio Redis, PubSub, cluster, pipelines/transactions, connection pools, and connection close/health-check paths to record operation durations, errors/retries, pubsub receive events, streaming lag, connection create/wait/close metrics, and geo failover metrics.

Updates async retry to optionally pass a failure count (with_failure_count) and uses it throughout async code to report retry attempts, while also extending sync streaming commands to export lag metrics for both sync and async (wrapping coroutine responses). Adds an async mode to benchmarks/otel_benchmark.py and new/updated asyncio tests validating metric emission and required connection metadata.

Written by Cursor Bugbot for commit 0e7d587. This will update automatically on new commits. Configure here.

vladvildanov and others added 12 commits February 17, 2026 11:55
* Added async recorder functions

* Added exception handling

* Added exception handling
* Added export of error metric

* Update redis/asyncio/connection.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Codestyle fixes

* Added proper retry handling for metric export

* Fixed internal flag and retry catching

* Removed unnecessary argument

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Added operation duration metric export

* Rename metric export methods

* Update redis/asyncio/cluster.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update tests/test_asyncio/test_cluster.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Moved metric export, fixed attribute name

* Applied comments

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Added connection advanced metrics

* Codestyle fixes
* Added pubsub metrics export

* Removed imports

* Changed assertion
* Added streaming lag metric export

* Update redis/commands/core.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Moved imports

* Fixed assertions

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@vladvildanov vladvildanov added the feature New feature label Feb 24, 2026
@jit-ci
Copy link
Copy Markdown

jit-ci Bot commented Feb 24, 2026

🛡️ Jit Security Scan Results

CRITICAL HIGH MEDIUM

✅ No security findings were detected in this PR


Security scan by Jit

Comment thread redis/asyncio/observability/recorder.py Outdated
Comment thread redis/asyncio/observability/recorder.py
Comment thread redis/asyncio/client.py
Comment thread redis/asyncio/client.py Outdated
Comment thread redis/asyncio/client.py
Comment thread redis/asyncio/cluster.py
Comment thread redis/asyncio/connection.py Outdated
Comment thread redis/asyncio/connection.py
Comment thread tests/test_asyncio/test_client.py
Comment thread tests/test_asyncio/test_cluster.py Outdated
Comment thread redis/asyncio/client.py
Copy link
Copy Markdown
Collaborator

@petyaslavova petyaslavova left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Just check if the cursor comment is valid, and I think it is ready for merging

Comment thread redis/asyncio/client.py Outdated
Comment thread redis/connection.py
Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Comment thread redis/commands/core.py
Comment thread redis/asyncio/client.py
@vladvildanov vladvildanov merged commit e9fba86 into master Mar 3, 2026
64 checks passed
@vladvildanov vladvildanov deleted the feat/async-observability branch March 3, 2026 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants