Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-43093

Concurrent calls to ShardingReplicaSetChangeListener::onConfirmedSet can cause starvation in the fixed executor

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical - P2
    • Resolution: Fixed
    • Affects Version/s: 4.2.0
    • Fix Version/s: 4.2.1, 4.3.1
    • Component/s: Upgrade/Downgrade
    • Labels:
    • Backwards Compatibility:
      Fully Compatible
    • Operating System:
      ALL
    • Backport Requested:
      v4.2

      Description

      The issue is that the callback is running in the fixed executor and then also tries doing a blocking call inside the fixed executor here. So if the change callback for different replica sets happened roughly at the same time, all of them will be blocked waiting for a worker thread to be available (which are none, since all of them are waiting for the same thing)

      Original description:

      Hi

      I setup the cluster with MongoDB 4.0 without problems, but  I am struggling to setup a cluster with MongoDB 4.2.

      As soon as I add more than 3 shards  to the cluster (I would like to have up to 24), I am not able to run a mongos anymore. It looks fine until the first restart of the mongos process. When I remove all shards and only keep 3 of them, everything is fine again. It does not matter which shard I keep, it seems something with the number of > 3. 

      I tried a lot of things, removed auth, removed encryption, removed compression: no luck.

      I cannot believe that this is a bug in 4.2.0 because even in the highest loglevel with exception tracing I cannot see any errors, the mongos is just not starting up until the point when it should start listening on all interfaces (with net.bindIpAll true) or on localhost (started with no special interface binding).

      Can you give me some help regarding this problem I am facing.

      Thank you

       

        Attachments

        1. mongo_shell.txt
          2 kB
        2. mongod_config_cmodb803.log
          34.54 MB
        3. mongod_config_cmodb803.zip
          2.59 MB
        4. mongos.log
          593 kB
        5. mongos.log.2019-08-30T11-20-05
          414 kB
        6. mongoshell.txt
          8 kB

          Issue Links

            Activity

              People

              Votes:
              2 Vote for this issue
              Watchers:
              18 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: