-
Type:
Bug
-
Resolution: Done
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Cluster Scalability
-
ALL
-
ClusterScalability 17Aug-31Aug
-
0
-
None
-
None
-
None
-
None
-
None
-
None
-
None
SERVER-131726 lowered cursorTimeoutMillis to 10s (and clientCursorMonitorFrequencySecs to 1s) in random_moveChunk_constraint_validation.js so that a dangling idle cursor left by the collMod constraint-upgrade scan is reaped quickly instead of blocking range deletions. The workaround is applied in $config.setup and restored in $config.teardown.
That window is too narrow. The lowered timeout only covers the workload body; it was scoped to the workload's own 60-second assert.soon. But CheckOrphansDeleted is a resmoke hook that runs after the workload, by which point teardown has already restored cursorTimeoutMillis to its 600000ms default. A cursor that goes idle near the end of the run is therefore reaped on the 10-minute default, racing the hook's own 10-minute budget.
From BF-45640 logs:
13:10:38.092 setParameter cursorTimeoutMillis 600000 -> 10000 (setup, all 6 shard nodes) 13:10:54.832 cursor 6903907431906337820 goes idle 13:10:55.292 RDELETER 6180600 "Range deletion will be scheduled after all possibly dependent queries finish" — cursorsDirectlyReferringTheNamespace: [6903907431906337820] 13:10:55.845 setParameter cursorTimeoutMillis 10000 -> 600000 (teardown, all 6 shard nodes) ... 13:20:56.772 clientcursormon "Cursor timed out" — 10 min, not 10 s 13:20:56.772 RDELETER 7536601 "Finished waiting for ongoing queries for range deletion task" 13:20:57.804 RDELETER 9239400 "Finished deletion of documents in orphan range" — 1009 docs, 32ms
The teardown restore beat the blocking cursor by 1.01 seconds. Under the lowered value the cursor would have been reaped at ~13:11:04.8 and the range deleted immediately after roughly 9 minutes inside the hook's budget. Instead the hook timed out at ~13:20:57 and failed the test on a range that was deleted milliseconds later. The range deleter itself is not at fault: once unblocked it completed in 32ms.
- is related to
-
SERVER-131726 Temporarily lower cursor timeout in random_moveChunk_constraint_validation.js
-
- Closed
-