tso: Fix keyspace-group split causing keyspace to fallback to group 0#9823
tso: Fix keyspace-group split causing keyspace to fallback to group 0#9823ti-chi-bot[bot] merged 2 commits intotikv:masterfrom
Conversation
|
Hi @ystaticy. Thanks for your PR. I'm waiting for a tikv member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/ok-to-test |
|
/test pull-unit-test-next-gen |
|
/retest |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #9823 +/- ##
==========================================
+ Coverage 76.82% 76.97% +0.14%
==========================================
Files 491 491
Lines 78440 78454 +14
==========================================
+ Hits 60261 60389 +128
+ Misses 14478 14388 -90
+ Partials 3701 3677 -24
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
/test pull-unit-test-next-gen |
|
@ystaticy: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: JmPotato, rleungx The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
What problem does this PR solve?
The keyspace tso group does not fall back to group 0 and is updated directly to the target group
Issue Number: Close #9822
What is changed and how does it work?
Save the split target keyspace group first, then save the split source keyspace group. The order matters: if we save splitSourceKg first (which removes some keyspaces from it), there will be a brief moment where those keyspaces don't belong to any group, causing them to fallback to group 0. By saving splitTargetKg first (which contains the split keyspaces), we ensure the keyspaces always belong to a valid group during the transition.
1.During the split process, save the target group first, then the source group. This avoids deleting the keyspace-to-group mapping from memory (caused by saving the source group first) and thus prevents falling back to group 0.
2. The watch process ensures order based on the revision, and the processing of putFn is also serial.
3. Updating the target group first does not trigger the issue of "premature" updates to the keyspace-group ID mapping. This is because during the split process, the client will keep retrying until the split is finally completed, after which the client can send requests normally.
4. The delayBeforeCheckingElectionMember failpoint is used to lengthen the split process, making it easier to reproduce the scenario where TSO requests are received between the two keyspace group save operations during the split.
Check List
Tests
Code changes
Side effects
Related changes
pingcap/docs/pingcap/docs-cn:pingcap/tiup:Release note