Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-34582

AsyncRequestsSender can block network threads during construction

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Critical - P2 Critical - P2
    • 3.6.5, 3.7.7
    • Affects Version/s: 3.6.4
    • Component/s: Networking
    • Labels:
    • Fully Compatible
    • ALL
    • v3.6
    • Platforms 2018-04-23, Platforms 2018-05-07
    • 0

      • The AsyncRequestsSender holds a lock during construction and work scheduling.
      • This lock prevents callbacks from running if their response comes back during scheduling.
      • Scheduling can take a long time (up to 20 seconds per shard) if a read preference cannot be satisfied. This is done by a blocking call into the ReplicaSetMonitor

      The bad sequence of events is:

      1. Scatter gather request to two shards is dispatched
      2. The first host suceeds in targetting and runs
      3. The second host cannot satisfy it's read pref, blocking holding a lock
      4. The first request suceeds, blocking in running _handleResponse

      If you have enough of those, you can saturate all background networking workers, making your mongos completely unresponsive until targeting can succeed.

            mira.carey@mongodb.com Mira Carey
            mira.carey@mongodb.com Mira Carey
            0 Vote for this issue
            8 Start watching this issue