Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-20559

Race condition in shard registry during concurrent sharding operations

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: 3.1.8
    • Fix Version/s: 3.1.9
    • Component/s: Sharding
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Steps To Reproduce:
      Hide

      This bug was uncovered through the FSM concurrency suite. To reproduce more simply, run moveChunk, splitChunk, and mergeChunk commands at a high degree of concurrency (>20 threads).

      Show
      This bug was uncovered through the FSM concurrency suite. To reproduce more simply, run moveChunk, splitChunk, and mergeChunk commands at a high degree of concurrency (>20 threads).
    • Sprint:
      TIG A (10/09/15)

      Description

      The moveChunk command routinely performs a reload of all shards in the shard registry, which clears the shard registry's ShardMap objects. The ShardMap objects contain shared pointers to Shard objects, so the Shard objects are deleted on these reloads.

      Other shard commands such as splitChunk and mergeChunks also obtain shared pointers to these Shard objects to grab the Shard's RemoteCommandTargeter object, which is owned by the Shard. The commands release the shared pointer to the Shard object but continue to use the RemoteCommandTargeter, so if the Shard is deleted during a concurrent moveChunk, then its RemoteCommandTargeter is deleted along with it, leaving the splitChunk or mergeChunk commands with an invalid reference to a deleted RemoteCommandTargeter. When they then attempt to use the RemoteCommandTargeter, a use-after-free occurs.

      Potential fix: remove the intermediate _targeter() method so that shared_ptr to the Shard is in scope for as long as the RemoteCommandTargeter.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              esha.maharishi Esha Maharishi
              Reporter:
              esha.maharishi Esha Maharishi
              Participants:
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: