Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-36443

Long-running queries should not cause a build-up of unused ChunkManager objects

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.14, 4.1.13, 4.0.11
    • Component/s: Sharding
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v4.0, v3.6
    • Sprint:
      Sharding 2019-05-06, Sharding 2019-05-20, Sharding 2019-06-03
    • Case:

      Description

      In SERVER-35666 we saw that keeping a query open for a long time can lead to memory buildup due to ChunkManager objects not being cleaned up. This is because here, when we go to expire old ChunkManager objects, we iterate through the queue of metadata objects from oldest to newest, and if we find an object with a usageCounter > 0, we stop, so that we don't delete an object used by an ongoing query.

      However, this is overly conservative. It's possible that the oldest object in the queue could have one query attached to it (usageCounter = 1), while many newer metadata objects have usageCounter = 0 and thus could be deleted. I believe we could also delete even metadata objects that have orphans on them, if the metadata object is never used by the range deleter.

        Attachments

          Issue Links

            Activity

              People

              • Votes:
                3 Vote for this issue
                Watchers:
                17 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: