[dr-autosync] scheduler resume after switching pd leader for 5min

[mayjiang0203]

Bug Report

What did you do?

After change pd leader, check scheduler status.

What did you expect to see?

Be better if take less than 2min.

What did you see instead?

After fix merge in 6.5.4, still hit same issue with #6920, but the fix for #6920 did reduce the probability of occurrence of this problem.

What version of PD are you using (pd-server -V)?

v6.5.4

[mayjiang0203]

/severity major
/assign @HuSharp

[HuSharp]

Assume this situation:
If pd1 is the leader, then transfer the leader to pd2. Assume the pd1 and pd2 sync index both are 100, as time passes there no sync index is updated, and then transfer the leader to pd1.

when pd2 is looking for pd1 to sync region, it will use the next index on pd1 and compare it with pd2.
But since pd1 and pd2 were already equal before, which means that the region will not be synchronized to pd1 anymore.
This un-sync action results in syncing region client will not put region's fromHeartbeat into false
which means in processRegionHeartbeat process will judge all regions IsNew is false and then can not collect all regions.
The failed collection resulted in Scheduling is blocked for around 5 mins.