cherry-pick: scheduler: graceful shutdown implement (#9720)

[hujiatao0]

close #9719

Add an is_stopping status to the StoreHeartbeat message. When TiKV receives a SIGTERM, it sets this flag. This change adds a new evict-stopping-store-scheduler to PD, which is analogous to the evict-slow-store-scheduler. It proactively transfers leaders away from nodes by inspecting the is_stopping status from store heartbeats.

What problem does this PR solve?

Issue Number: Close #9719

What is changed and how does it work?

Add an is_stopping status to the StoreHeartbeat message. When TiKV receives a SIGTERM, it sets this flag. This change adds a new evict-stopping-store-scheduler to PD, which is analogous to the evict-slow-store-scheduler. It proactively transfers leaders away from nodes by inspecting the is_stopping status from store heartbeats.

Check List

Tests

• Unit test

Code changes

• Has the configuration change

Side effects

• Increased code complexity

Related changes

• PR to update pingcap/docs/pingcap/docs-cn:

Release note

None.

[ti-chi-bot]

Hi @hujiatao0. Thanks for your PR.

I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[ti-chi-bot]

@hujiatao0: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[codecov]

Codecov Report

❌ Patch coverage is 66.96970% with 109 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.59%. Comparing base (d190c0e) to head (19b9481).
⚠️ Report is 1 commits behind head on release-8.5-20250929-v8.5.0.

❌ Your patch status has failed because the patch coverage (66.96%) is below the target coverage (74.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@                       Coverage Diff                       @@
##           release-8.5-20250929-v8.5.0    #9790      +/-   ##
===============================================================
- Coverage                        75.86%   75.59%   -0.28%     
===============================================================
  Files                              462      463       +1     
  Lines                            72355    72684     +329     
===============================================================
+ Hits                             54893    54943      +50     
- Misses                           13981    14224     +243     
- Partials                          3481     3517      +36     

Flag	Coverage Δ
unittests	75.59% <66.96%> (-0.28%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[hujiatao0]

cc @rleungx @okJiang @niubell

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2025-09-30 06:12:50.988200654 +0000 UTC m=+151551.244932044: ☑️ agreed by okJiang.
• 2025-09-30 06:22:57.238367388 +0000 UTC m=+152157.495098788: ☑️ agreed by rleungx.

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: niubell, okJiang, rleungx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [rleungx]
• pkg/schedule/config/OWNERS [niubell]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment