`mypyc runtime tests with py39-macos` is unstable

[WeilerMarcel]

Bug Report

The following tests in mypyc runtime tests with py39-macos spuriously failed during PR #14934 , even though I did not touch any code related to them:

• testIncrementalCompilation1_separate
• testIncrementalCompilation4_separate

I added two empty dummy commits to get the pipeline to succeed. The following three commits have no code changes, but for the first two some of the tests fail:

• 32389c1 Two failing tests: https://github.com/python/mypy/actions/runs/6786316124/job/18446597173?pr=14934
• 8142772 One failing test: https://github.com/python/mypy/actions/runs/6786818504/job/18448203144?pr=14934
• bc42c57 No failing tests: https://github.com/python/mypy/actions/runs/6796064753/job/18475383911?pr=14934

To Reproduce

Trigger a CI pipeline run without code changes.

Expected Behavior

The test runs are consistent.

Actual Behavior

Tests spuriously fail. See links to the log files above.

Your Environment

This happened in the CI runs of !PR #14934.

[JukkaL]

Yeah, the macOS mypyc runtime tests have been flaky for some time now. We have some hypotheses about what might be going on, but as far as I know this still bit of a mystery.

[hauntsaninja]

I wasn't aware we had hypotheses! The only thing I confirmed was that it doesn't seem to be related to any code change in mypy, something changed out from underneath us