Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-26740

Total connections is not stable and spiky when user authentication is enabled

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Duplicate
    • Affects Version/s: 3.2.10
    • Fix Version/s: None
    • Component/s: Networking, Security
    • Labels:
      None
    • Operating System:
      ALL
    • Case:

      Description

      MongoDB cluster is consist of 28 shards and 15 mongos.
      There's about 4K/sec queries per shard and small DMLs and there's about 600~1000 connections(from mongos) per shard.

      MongoDB connections graph is spiky only when user authentication is enabled Not with disabled authentication. And java client driver return errors(like below) or slow-responses when user authentication is enabled. (I attached connections and query/sec graphs).

      [ERROR] [c.k.s.m.g.r.RequestExecuteCallable] execute (71): Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 400 has been exceeded.
      com.mongodb.MongoWaitQueueFullException: Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 400 has been exceeded.

      Not exactly 5 minutes, but connection spike(and client queue error) is happened about 5 minutes intervals.
      I am not sure "ConnectionPool::kDefaultHostTimeout" cause this issue yet, But I think 5 minutes is too short to shutdown connection pool. it seems so aggressive. Is there any reason to choose 5 minutes to shutdown pool ?

      https://github.com/mongodb/mongo/blob/master/src/mongo/executor/connection_pool.cpp#L177

        Attachments

        1. Connections.png
          Connections.png
          38 kB
        2. diagnostic.data.tar.gz
          10.85 MB
        3. Query.png
          Query.png
          45 kB

          Issue Links

            Activity

              People

              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: