Skip to content

Make SegmentSelectorEditor tests less flaky#24138

Merged
chippison merged 2 commits into5.x-devfrom
segment_selector_editor_flaky
Feb 26, 2026
Merged

Make SegmentSelectorEditor tests less flaky#24138
chippison merged 2 commits into5.x-devfrom
segment_selector_editor_flaky

Conversation

@caddoo
Copy link
Copy Markdown
Contributor

@caddoo caddoo commented Feb 26, 2026

Description

These screenshot tests were flaky for me (each failed at least once), likely due to hover/tooltips being captured.
This PR moves the mouse out of the capture area before taking the affected screenshots to stabilize them.

It also improves debugging clarity by splitting a shared baseline: two tests previously used updated.png, which made
failures harder to interpret (often showing no useful diff). The reload-step test now uses its own duplicated baseline
image so each assertion can fail independently with clearer output.

Failures addressed:
https://builds-artifacts.matomo.org/matomo-org/matomo/preview_alerts/22417472720/

https://builds-artifacts.matomo.org/matomo-org/matomo/segment_selector_editor_flaky/22423974975/

Checklist

  • [✔] I have understood, reviewed, and tested all AI outputs before use
  • [✔] All AI instructions respect security, IP, and privacy rules

Review

@caddoo caddoo added this to the 5.8.0 milestone Feb 26, 2026
@caddoo caddoo added the not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org. label Feb 26, 2026
@caddoo caddoo changed the title Make SegmentSelectorEditor tests less flakey Make SegmentSelectorEditor tests less flaky Feb 26, 2026
@caddoo caddoo marked this pull request as ready for review February 26, 2026 04:24
@caddoo caddoo requested a review from a team February 26, 2026 04:48
@sgiehl sgiehl modified the milestones: 5.8.0, 5.9.0 Feb 26, 2026
@chippison chippison merged commit e737c6d into 5.x-dev Feb 26, 2026
33 checks passed
@chippison chippison deleted the segment_selector_editor_flaky branch February 26, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

not-in-changelog For issues or pull requests that should not be included in our release changelog on matomo.org.

Development

Successfully merging this pull request may close these issues.

3 participants