Description
One of our e2e tests is failing due to a mismatched screenshot. This is not currently reproducible locally.
1) [webkit] › table-operations.spec.ts:369:5 › custom column › Drag ──────────────────────────────
Error: Screenshot comparison failed:
4289 pixels (ratio 0.02 of all image pixels) are different.
Call log:
- expect.toHaveScreenshot with timeout 15000ms
- verifying given screenshot expectation
- waiting for locator('.iris-grid-column')
- locator resolved to <div class="iris-grid-column">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for element to be visible and stable
- element is visible and stable
- 4289 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for locator('.iris-grid-column')
- locator resolved to <div class="iris-grid-column">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for element to be visible and stable
- element is visible and stable
- captured a stable screenshot
- 4289 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/web-client-ui/web-client-ui/tests/table-operations.spec.ts-snapshots/custom-column-3-webkit-linux.png
Received: /home/runner/work/web-client-ui/web-client-ui/test-results/table-operations-custom-column-webkit/custom-column-3-actual.png
Diff: /home/runner/work/web-client-ui/web-client-ui/test-results/table-operations-custom-column-webkit/custom-column-3-diff.png
444 |
445 | await waitForLoadingDone(page);
> 446 | await expect(page.locator('.iris-grid-column')).toHaveScreenshot();
| ^
447 | });
448 | });
449 |
at /home/runner/work/web-client-ui/web-client-ui/tests/table-operations.spec.ts:446:53
at /home/runner/work/web-client-ui/web-client-ui/tests/table-operations.spec.ts:4[17](https://github.com/deephaven/web-client-ui/actions/runs/6331018335/job/17194694692#step:9:18):3
attachment #1: custom-column-3-expected.png (image/png) ────────────────────────────────────────
tests/table-operations.spec.ts-snapshots/custom-column-3-webkit-linux.png
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #2: custom-column-3-actual.png (image/png) ──────────────────────────────────────────
test-results/table-operations-custom-column-webkit/custom-column-3-actual.png
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #3: custom-column-3-diff.png (image/png) ────────────────────────────────────────────
test-results/table-operations-custom-column-webkit/custom-column-3-diff.png
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #4: video (video/webm) ──────────────────────────────────────────────────────────────
test-results/table-operations-custom-column-webkit/video.webm
────────────────────────────────────────────────────────────────────────────────────────────────
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Screenshot comparison failed:
4289 pixels (ratio 0.02 of all image pixels) are different.
Call log:
- expect.toHaveScreenshot with timeout 15000ms
- verifying given screenshot expectation
- waiting for locator('.iris-grid-column')
- locator resolved to <div class="iris-grid-column">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for element to be visible and stable
- element is visible and stable
- 4[28](https://github.com/deephaven/web-client-ui/actions/runs/6331018335/job/17194694692#step:9:29)9 pixels (ratio 0.02 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for locator('.iris-grid-column')
- locator resolved to <div class="iris-grid-column">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for element to be visible and stable
- element is visible and stable
- captured a stable screenshot
- 4289 pixels (ratio 0.02 of all image pixels) are different.
Expected: /home/runner/work/web-client-ui/web-client-ui/tests/table-operations.spec.ts-snapshots/custom-column-3-webkit-linux.png
Received: /home/runner/work/web-client-ui/web-client-ui/test-results/table-operations-custom-column-webkit-retry1/custom-column-3-actual.png
Diff: /home/runner/work/web-client-ui/web-client-ui/test-results/table-operations-custom-column-webkit-retry1/custom-column-3-diff.png
444 |
445 | await waitForLoadingDone(page);
> 446 | await expect(page.locator('.iris-grid-column')).toHaveScreenshot();
| ^
447 | });
448 | });
449 |
at /home/runner/work/web-client-ui/web-client-ui/tests/table-operations.spec.ts:446:53
at /home/runner/work/web-client-ui/web-client-ui/tests/table-operations.spec.ts:417:3
attachment #1: custom-column-3-expected.png (image/png) ────────────────────────────────────────
tests/table-operations.spec.ts-snapshots/custom-column-3-webkit-linux.png
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #2: custom-column-3-actual.png (image/png) ──────────────────────────────────────────
test-results/table-operations-custom-column-webkit-retry1/custom-column-3-actual.png
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #3: custom-column-3-diff.png (image/png) ────────────────────────────────────────────
test-results/table-operations-custom-column-webkit-retry1/custom-column-3-diff.png
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #4: video (video/webm) ──────────────────────────────────────────────────────────────
test-results/table-operations-custom-column-webkit-retry1/video.webm
────────────────────────────────────────────────────────────────────────────────────────────────
attachment #5: trace (application/zip) ─────────────────────────────────────────────────────────
test-results/table-operations-custom-column-webkit-retry1/trace.zip
Usage:
npx playwright show-trace test-results/table-operations-custom-column-webkit-retry1/trace.zip
────────────────────────────────────────────────────────────────────────────────────────────────
Slow test file: [webkit] › table-operations.spec.ts (3.6m)
Slow test file: [chromium] › table-operations.spec.ts (1.3m)
Slow test file: [firefox] › table-operations.spec.ts (1.2m)
Slow test file: [webkit] › table.spec.ts (37.9s)
Slow test file: [webkit] › notebook.spec.ts ([30](https://github.com/deephaven/web-client-ui/actions/runs/6331018335/job/17194694692#step:9:31).6s)
Consider splitting slow test files to speed up parallel execution
1 failed
[webkit] › table-operations.spec.ts:[36](https://github.com/deephaven/web-client-ui/actions/runs/6331018335/job/17194694692#step:9:37)9:5 › custom column ──────────────────────────────────────
2 skipped
[69](https://github.com/deephaven/web-client-ui/actions/runs/6331018335/job/17194694692#step:9:70) passed (9.9m)
Error: Process completed with exit code 1.
Steps to reproduce
This has been failing consistently for new PRs
https://github.com/deephaven/web-client-ui/actions/workflows/e2e.yml
Have not yet been able to reproduce locally
Expected results
Test should pass
Actual results
Test fails
Additional details and attachments
Versions
- Engine Version: ...
- Web UI Version: ...
- Java Version: ...
- Barrage Version: ...
- OS: ...
- Browser: ...
- Docker: ...
Description
One of our e2e tests is failing due to a mismatched screenshot. This is not currently reproducible locally.
Steps to reproduce
This has been failing consistently for new PRs
https://github.com/deephaven/web-client-ui/actions/workflows/e2e.yml
Have not yet been able to reproduce locally
Expected results
Test should pass
Actual results
Test fails
Additional details and attachments
Versions