-
Type:
Bug
-
Resolution: Duplicate
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Cluster Scalability
-
ALL
-
v9.0
-
ClusterScalability 20Jul-3Aug, ClusterScalability 3Aug-17Aug
-
200
-
None
-
None
-
None
-
None
-
None
-
None
-
None
The resharding change streams monitor can starve the donor state machine for many minutes (see BF-44929).
The monitor consumes change events in a synchronous loop that runs on the ReshardingDonorService executor, which has only 2 threads by default. A batch only ends on the final event, the batch size limit, or the batch time limit (default 5 minutes). On an idle collection that means one monitor pins an executor thread for the full 5 minutes.
In the failing runs the balancer's random moveCollections keep aborting with DuplicateKey. The aborted operation's monitor would keep running and pin one thread, then the next moveCollection's monitor would pin the other. That operation's donor state transition would then stay queued for ~300s while holding the collection DDL lock. The DDL lock timeout is also 5 minutes, so a concurrent dropIndexes fails with LockBusy if it starts waiting just after the reshard takes the lock.
This started with featureFlagReshardingVerification being enabled (SERVER-94478).
- duplicates
-
SERVER-132613 Resharding recipient, donor and coordinator services can destroy actively used cancellable opCtx factory preventing cancellation signal propagation
-
- Closed
-
- is related to
-
SERVER-132613 Resharding recipient, donor and coordinator services can destroy actively used cancellable opCtx factory preventing cancellation signal propagation
-
- Closed
-
-
SERVER-94478 Enable SPM-3918 feature flag
-
- Closed
-