Skip to content

Commit 94a36e1

Browse files
committed
Add task and worker lifecycle metrics to Celery instrumentation
Introduce Flower-compatible metrics for Celery task and worker events: - flower.events.total: counter for task-sent, task-received, task-started, task-succeeded, task-failed, task-retried, task-revoked - flower.task.runtime.seconds: histogram of task execution time - flower.worker.number.of.currently.executing.tasks: gauge of in-flight tasks per worker - flower.worker.online: gauge tracking worker online/offline status CeleryInstrumentor (child process) handles task tracing and task-level metrics. CeleryWorkerInstrumentor (main process) handles worker lifecycle signals (worker_ready, worker_shutdown) and main-process events (task_received, task_revoked). Prefetch-time metrics (task_prefetch_time_seconds, worker_prefetched_tasks) are intentionally omitted — Celery splits task_received and task_prerun across processes in prefork mode, making it impossible to compute the delta without external event consumption.
1 parent 34bfc28 commit 94a36e1

9 files changed

Lines changed: 1670 additions & 80 deletions

File tree

CHANGELOG.md

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1515

1616
- Bump `pylint` to `4.0.5`
1717
([#4244](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4244))
18+
- `opentelemetry-instrumentation-celery`: Add task and worker lifecycle metrics matching Celery Flower
19+
([#4439](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4439))
20+
21+
### Fixed
22+
23+
- `opentelemetry-instrumentation-celery`: Fix memory leak in `task_id_to_start_time` dict never being cleaned up
24+
([#4439](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/4439))
1825

1926
### Breaking changes
2027

docs/nitpick-exceptions.ini

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,8 @@ py-class=
4747
fastapi.applications.FastAPI
4848
starlette.applications.Starlette
4949
_contextvars.Token
50+
celery.worker.request.Request
51+
opentelemetry.metrics.Gauge
5052

5153
any=
5254
; API

instrumentation/opentelemetry-instrumentation-celery/pyproject.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,7 @@ instruments = [
3737

3838
[project.entry-points.opentelemetry_instrumentor]
3939
celery = "opentelemetry.instrumentation.celery:CeleryInstrumentor"
40+
celery_worker = "opentelemetry.instrumentation.celery:CeleryWorkerInstrumentor"
4041

4142
[project.urls]
4243
Homepage = "https://github.com/open-telemetry/opentelemetry-python-contrib/tree/main/instrumentation/opentelemetry-instrumentation-celery"

0 commit comments

Comments
 (0)