Commit 89a8172
committed
Fix test_shutdown_worker leaving stale callback that breaks later tests
Root cause:
test_shutdown_worker was an async test (@pytest.mark.asyncio), so
pytest-asyncio ran it via loop.run_until_complete(). Inside the test,
shutdown_worker() calls runner.close() -> loop.stop(). Calling
loop.stop() from within loop.run_until_complete() leaves a stale
_run_until_complete_cb in the event loop's ready queue. This callback
calls loop.stop() when it fires, which poisons the next
run_until_complete() call on the same loop with:
RuntimeError: Event loop stopped before Future completed
Why the next test is affected:
The event_loop fixture in conftest.py returns manager.get_runner().loop.
After test_shutdown_worker, this loop still has the stale callback.
When a later test (e.g. test_calc_job_node_get_builder_restart) calls
run_until_complete() on the same loop, the stale callback fires
loop.stop() prematurely.
Why the _reset_runner autouse fixture masked the issue:
The _reset_runner fixture called manager.reset_runner() after every
test, which called runner.close() -> loop.close(). This closed the
loop entirely, so get_or_create_event_loop() created a fresh loop
for the next test without the stale callback. Removing this fixture
exposed the underlying bug.
Why run_forever fixes it:
In the real daemon, start_daemon_worker() uses loop.run_forever(),
not loop.run_until_complete(). run_forever() does not add a
_run_until_complete_cb, so loop.stop() cleanly exits the loop with
no stale callbacks. The fix changes the test to use the same pattern:
schedule shutdown_worker as a task, then call run_forever(). This
mirrors production behavior and avoids the stale callback.1 parent a13f4e5 commit 89a8172
2 files changed
Lines changed: 4 additions & 15 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
55 | 55 | | |
56 | 56 | | |
57 | 57 | | |
58 | | - | |
59 | | - | |
60 | | - | |
61 | | - | |
62 | | - | |
63 | | - | |
64 | 58 | | |
65 | 59 | | |
66 | 60 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | | - | |
| 19 | + | |
21 | 20 | | |
22 | 21 | | |
23 | | - | |
24 | | - | |
25 | | - | |
26 | | - | |
27 | | - | |
28 | | - | |
29 | | - | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
30 | 25 | | |
31 | 26 | | |
32 | 27 | | |
| |||
0 commit comments