Refactor `Gauge` metrics to be homeserver-scoped by MadLittleMods · Pull Request #18725 · element-hq/synapse

MadLittleMods · 2025-07-23T22:46:51Z

Bulk refactor Gauge metrics to be homeserver-scoped. We also add lints to make sure that new Gauge metrics don't sneak in without using the server_name label (SERVER_NAME_LABEL).

Part of #18592

Testing strategy

Add the metrics listener in your homeserver.yaml

listeners:
  # This is just showing how to configure metrics either way
  #
  # `http` `metrics` resource
  - port: 9322
    type: http
    bind_addresses: ['127.0.0.1']
    resources:
      - names: [metrics]
        compress: false
  # `metrics` listener
  - port: 9323
    type: metrics
    bind_addresses: ['127.0.0.1']

Start the homeserver: poetry run synapse_homeserver --config-path homeserver.yaml
Fetch http://localhost:9322/_synapse/metrics and/or http://localhost:9323/metrics
Observe response includes the TODO metrics with the server_name label

Pull Request Checklist

Pull request is based on the develop branch
Pull request includes a changelog file. The entry should:
- Be a short description of your change which makes sense to users. "Fixed a bug that prevented receiving messages from other servers." instead of "Moved X method from EventStore to EventWorkerStore.".
- Use markdown where necessary, mostly for code blocks.
- End with either a period (.) or an exclamation mark (!).
- Start with a capital letter.
- Feel free to credit yourself, by adding a sentence "Contributed by @github_username." or "Contributed by [Your Name]." to the end of the entry.
Code style is correct (run the linters)

From #18656

``` synapse/metrics/__init__.py:522: error: Expected the `labelnames` argument of Histogram to be a list of label names (including `SERVER_NAME_LABEL`), but got TupleExpr:528( StrExpr(type) StrExpr(reason) NameExpr(SERVER_NAME_LABEL [synapse.metrics.SERVER_NAME_LABEL])). If this is a process-level metric (vs homeserver-level), use a type ignore comment to disable this check. [missing-server-name-label] ```

MadLittleMods · 2025-07-23T22:48:17Z

    "synapse_federation_last_received_pdu_time",
    "The timestamp of the last PDU which was successfully received from the given domain",
-    labelnames=("server_name",),
+    labelnames=("origin_server_name", SERVER_NAME_LABEL),


This metric was already using the server_name label so I've had to rename the label. I've updated the contrib/grafana/synapse.json but it's something we probably want to call out in the upgrade notes (added)

MadLittleMods · 2025-07-23T22:48:20Z

    "synapse_federation_last_sent_pdu_time",
    "The timestamp of the last PDU which was successfully sent to the given domain",
-    labelnames=("server_name",),
+    labelnames=("destination_server_name", SERVER_NAME_LABEL),


This metric was already using the server_name label so I've had to rename the label. I've updated the contrib/grafana/synapse.json but it's something we probably want to call out in the upgrade notes (added)

devonh

Other than the small fixes and the comment about argument consistency, these changes seem pretty straightforward.

devonh · 2025-07-24T22:29:22Z

        return None


+def check_prometheus_metric_instantiation(ctx: FunctionSigContext) -> CallableType:


Thank you for adding this lint!
This should make maintenance of these changes much more manageable going forward.

Conflicts: scripts-dev/mypy_synapse_plugin.py

See #18725 (comment)

MadLittleMods · 2025-07-29T15:38:22Z

Thanks for the review @devonh 🐬

MadLittleMods added 14 commits July 23, 2025 17:08

Add in base linting for metrics

b14eed7

From #18656

Fill in synapse/app/phone_stats_home.py

4bb84a6

Fill in synapse/federation/federation_server.py

1504100

Fill in synapse/federation/sender/transaction_manager.py

608f72e

Fill in synapse/metrics/__init__.py

0a2877e

Fill in synapse/metrics/_gc.py

0c93d85

Fill in synapse/metrics/common_usage_metrics.py

fe1a16f

Fill in synapse/push/pusherpool.py

944df9c

Fill in synapse/replication/http/_base.py

54e2374

Fill in synapse/storage/databases/main/event_federation.py

883062b

Fill in synapse/storage/databases/main/events_worker.py

90efa41

Fill in synapse/util/batching_queue.py

f4b6d35

Fill in synapse/util/caches/deferred_cache.py

656c3ad

MadLittleMods added the A-Metrics label Jul 23, 2025

MadLittleMods mentioned this pull request Jul 23, 2025

Refactor metrics to be scoped to the homeserver #18592

Closed

38 tasks

MadLittleMods commented Jul 23, 2025

View reviewed changes

Add changelog

c55e615

MadLittleMods force-pushed the madlittlemods/18592-refactor-gauge branch from 18de228 to 6b308e6 Compare July 23, 2025 22:56

Add upgrade notes

0fd34a6

MadLittleMods force-pushed the madlittlemods/18592-refactor-gauge branch from 6b308e6 to 0fd34a6 Compare July 23, 2025 22:56

github-actions Bot deployed to PR Documentation Preview July 23, 2025 22:58 Active

Fix make_fake_db_pool

563f543

github-actions Bot deployed to PR Documentation Preview July 23, 2025 23:27 Active

Fix event_persisted_position usage

554d588

github-actions Bot deployed to PR Documentation Preview July 23, 2025 23:39 Active

Fix lints

2536aaf

github-actions Bot deployed to PR Documentation Preview July 23, 2025 23:58 Active

Fill in event Gauge metrics from synapse/metrics/__init__.py

7b55ffb

github-actions Bot deployed to PR Documentation Preview July 24, 2025 00:18 Active

Merge branch 'develop' into madlittlemods/18592-refactor-gauge

68061f9

github-actions Bot deployed to PR Documentation Preview July 24, 2025 16:27 Active

MadLittleMods marked this pull request as ready for review July 24, 2025 18:42

MadLittleMods requested a review from a team as a code owner July 24, 2025 18:42

Remove debug log

d587aa9

github-actions Bot deployed to PR Documentation Preview July 24, 2025 20:37 Active

devonh reviewed Jul 24, 2025

View reviewed changes

Merge branch 'develop' into madlittlemods/18592-refactor-gauge

def4eb5

Conflicts: scripts-dev/mypy_synapse_plugin.py

github-actions Bot deployed to PR Documentation Preview July 25, 2025 20:02 Active

MadLittleMods added a commit that referenced this pull request Jul 25, 2025

Fix grammar

202ef03

See #18725 (comment)

MadLittleMods added a commit that referenced this pull request Jul 25, 2025

Fix duplicate word typo

26eef52

See #18725 (comment)

MadLittleMods added 2 commits July 25, 2025 15:23

Fix grammar

650ce32

See #18725 (comment)

Fix duplicate word typo

4387262

See #18725 (comment)

github-actions Bot deployed to PR Documentation Preview July 25, 2025 20:24 Active

MadLittleMods requested a review from devonh July 25, 2025 20:37

devonh approved these changes Jul 29, 2025

View reviewed changes

MadLittleMods merged commit f13a136 into develop Jul 29, 2025
78 of 80 checks passed

MadLittleMods deleted the madlittlemods/18592-refactor-gauge branch July 29, 2025 15:38

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor `Gauge` metrics to be homeserver-scoped#18725

Refactor `Gauge` metrics to be homeserver-scoped#18725
MadLittleMods merged 25 commits intodevelopfrom
madlittlemods/18592-refactor-gauge

MadLittleMods commented Jul 23, 2025 •

edited

Loading

Uh oh!

MadLittleMods Jul 23, 2025 •

edited

Loading

Uh oh!

MadLittleMods Jul 23, 2025 •

edited

Loading

Uh oh!

devonh left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devonh Jul 24, 2025

Uh oh!

Uh oh!

MadLittleMods commented Jul 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		return None


		def check_prometheus_metric_instantiation(ctx: FunctionSigContext) -> CallableType:

Conversation

MadLittleMods commented Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Testing strategy

Pull Request Checklist

Uh oh!

MadLittleMods Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MadLittleMods Jul 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

devonh left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

devonh Jul 24, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

MadLittleMods commented Jul 29, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MadLittleMods commented Jul 23, 2025 •

edited

Loading

MadLittleMods Jul 23, 2025 •

edited

Loading

MadLittleMods Jul 23, 2025 •

edited

Loading