random_moveChunk_constraint_validation.js: Restore cursor timeout before draining range deletions

XMLWordPrintableJSON

    • Type: Bug
    • Resolution: Done
    • Priority: Major - P3
    • 9.1.0-rc0
    • Affects Version/s: None
    • Component/s: None
    • None
    • Cluster Scalability
    • ALL
    • ClusterScalability 17Aug-31Aug
    • 0
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      SERVER-131726 lowered cursorTimeoutMillis to 10s (and clientCursorMonitorFrequencySecs to 1s) in random_moveChunk_constraint_validation.js so that a dangling idle cursor left by the collMod constraint-upgrade scan is reaped quickly instead of blocking range deletions. The workaround is applied in $config.setup and restored in $config.teardown.

      That window is too narrow. The lowered timeout only covers the workload body; it was scoped to the workload's own 60-second assert.soon. But CheckOrphansDeleted is a resmoke hook that runs after the workload, by which point teardown has already restored cursorTimeoutMillis to its 600000ms default. A cursor that goes idle near the end of the run is therefore reaped on the 10-minute default, racing the hook's own 10-minute budget.

      From BF-45640 logs:

      13:10:38.092  setParameter cursorTimeoutMillis 600000 -> 10000   (setup, all 6 shard nodes)
      13:10:54.832  cursor 6903907431906337820 goes idle
      13:10:55.292  RDELETER 6180600 "Range deletion will be scheduled after all possibly dependent
                    queries finish" — cursorsDirectlyReferringTheNamespace: [6903907431906337820]
      13:10:55.845  setParameter cursorTimeoutMillis 10000 -> 600000   (teardown, all 6 shard nodes)
      ...
      13:20:56.772  clientcursormon "Cursor timed out" — 10 min, not 10 s
      13:20:56.772  RDELETER 7536601 "Finished waiting for ongoing queries for range deletion task"
      13:20:57.804  RDELETER 9239400 "Finished deletion of documents in orphan range" — 1009 docs, 32ms
      

      The teardown restore beat the blocking cursor by 1.01 seconds. Under the lowered value the cursor would have been reaped at ~13:11:04.8 and the range deleted immediately after roughly 9 minutes inside the hook's budget. Instead the hook timed out at ~13:20:57 and failed the test on a range that was deleted milliseconds later. The range deleter itself is not at fault: once unblocked it completed in 32ms.

            Assignee:
            Nandini Bhartiya
            Reporter:
            Nandini Bhartiya
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: