Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-28822

Improve DBConnectionPool's growth semantics

    XMLWordPrintable

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.6.3, 3.7.2
    • Component/s: Networking
    • Labels:
      None
    • Backwards Compatibility:
      Fully Compatible
    • Backport Requested:
      v3.6, v3.4, v3.2
    • Sprint:
      Platforms 2017-09-11, Platforms 2017-10-02, Platforms 2017-12-04, Platforms 2017-12-18, Platforms 2018-01-01, Platforms 2018-01-15, Platforms 2018-02-12
    • Case:
    • Linked BF Score:
      0

      Description

      The DBConnectionPool (legacy connection pool that backs dbclient, used for write operations through 3.5.x) has hard-to-understand and occasionally problematic growth semantics.

      Our documentation for the knob that controls this pool is correct:

      Set the maximum size of the connection pools for outgoing connections to other mongod instances. The size of a pool does not prevent the creation of additional connections, but does prevent a connection pool from retaining connections in excess of the value of connPoolMaxConnsPerHost.

      However, it does not explain that connections retained in the pool will remain open in the pool forever, leading to a steady growth in long-lived outgoing server connections, then a plateau once the max pool size is reached, even when there is no load on the system.

      We could do the following things to improve DBConnectionPool's semantics and offer more control over its growth:

      • add a "MaxOpenConnections" option that would cap the number of connections that are simultaneously created (not just retained) by DBConnectionPool. For backwards compatibility this would default to infinity.
      • add an "IdleHostTimeoutMS" option, so that DBConnectionPool dumps its connections to a host after this amount of time spent idle, instead of keeping them open forever. For backwards compatibility this would default to "no timeout."
      • strongly suggest that customers with large clusters or many mongos set the connPoolMaxConnsPerHost and connPoolMaxShardedConnsPerHost parameters to some number that is much smaller than the default of 200.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jason.carey Jason Carey
              Reporter:
              samantha.ritter Samantha Ritter
              Participants:
              Votes:
              1 Vote for this issue
              Watchers:
              21 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: