Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-34582

AsyncRequestsSender can block network threads during construction

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical - P2
    • Resolution: Fixed
    • 3.6.4
    • 3.6.5, 3.7.7
    • Networking
    • None
    • Fully Compatible
    • ALL
    • v3.6
    • Platforms 2018-04-23, Platforms 2018-05-07
    • 0

    Description

      • The AsyncRequestsSender holds a lock during construction and work scheduling.
      • This lock prevents callbacks from running if their response comes back during scheduling.
      • Scheduling can take a long time (up to 20 seconds per shard) if a read preference cannot be satisfied. This is done by a blocking call into the ReplicaSetMonitor

      The bad sequence of events is:

      1. Scatter gather request to two shards is dispatched
      2. The first host suceeds in targetting and runs
      3. The second host cannot satisfy it's read pref, blocking holding a lock
      4. The first request suceeds, blocking in running _handleResponse

      If you have enough of those, you can saturate all background networking workers, making your mongos completely unresponsive until targeting can succeed.

      Attachments

        Issue Links

          Activity

            People

              jason.carey@mongodb.com Jason Carey
              jason.carey@mongodb.com Jason Carey
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: