Uploaded image for project: 'Documentation'
  1. Documentation
  2. DOCS-11281

Improve DBConnectionPool's growth semantics (SERVER-28822)

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.7.2, 3.6.3
    • Component/s: Server
    • Labels:
      None
    • # Replies:
      2
    • Last comment by Customer:
      true
    • Story Points:
      0.5

      Description

      Documentation Request Summary:

      This change adds new options and behavior for the dbclient pool, I will add a comment about them.

      ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
          maxInUseConnsPerHostParameter(ServerParameterSet::getGlobal(),
                                        "connPoolMaxInUseConnsPerHost",
                                        &ConnPoolOptions::maxInUseConnsPerHost);
       
      ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
          maxShardedInUseConnsPerHostParameter(ServerParameterSet::getGlobal(),
                                               "connPoolMaxShardedInUseConnsPerHost",
                                               &ConnPoolOptions::maxShardedInUseConnsPerHost);
       
      ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
          globalConnPoolIdleTimeoutParameter(ServerParameterSet::getGlobal(),
                                             "globalConnPoolIdleTimeoutMinutes",
                                             &ConnPoolOptions::globalConnPoolIdleTimeout);
       
      ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
          shardedConnPoolIdleTimeoutParameter(ServerParameterSet::getGlobal(),
                                              "shardedConnPoolIdleTimeoutMinutes",
                                              &ConnPoolOptions::shardedConnPoolIdleTimeout);
      
      

      Scope of changes:

      • parameters
      • 4.0.txt

      Impact to other docs outside of this product:

      none

      MVP:

      Resources:

      Engineering Ticket Description:

      The DBConnectionPool (legacy connection pool that backs dbclient, used for write operations through 3.5.x) has hard-to-understand and occasionally problematic growth semantics.

      Our documentation for the knob that controls this pool is correct:

      Set the maximum size of the connection pools for outgoing connections to other mongod instances. The size of a pool does not prevent the creation of additional connections, but does prevent a connection pool from retaining connections in excess of the value of connPoolMaxConnsPerHost.

      However, it does not explain that connections retained in the pool will remain open in the pool forever, leading to a steady growth in long-lived outgoing server connections, then a plateau once the max pool size is reached, even when there is no load on the system.

      We could do the following things to improve DBConnectionPool's semantics and offer more control over its growth:

      • add a "MaxOpenConnections" option that would cap the number of connections that are simultaneously created (not just retained) by DBConnectionPool. For backwards compatibility this would default to infinity.
      • add an "IdleHostTimeoutMS" option, so that DBConnectionPool dumps its connections to a host after this amount of time spent idle, instead of keeping them open forever. For backwards compatibility this would default to "no timeout."
      • strongly suggest that customers with large clusters or many mongos set the connPoolMaxConnsPerHost and connPoolMaxShardedConnsPerHost parameters to some number that is much smaller than the default of 200.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              kay.kim Kay Kim (Inactive)
              Reporter:
              kay.kim Kay Kim (Inactive)
              Participants:
              Last commenter:
              Caleb Thompson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved:
                Days since reply:
                2 years, 9 weeks, 3 days ago
                Date of 1st Reply: