[DOCS-11281] Improve DBConnectionPool's growth semantics (SERVER-28822) Created: 05/Feb/18  Updated: 29/Oct/23  Resolved: 08/Jun/18

Status: Closed
Project: Documentation
Component/s: Server
Affects Version/s: None
Fix Version/s: 3.7.2, 3.6.3

Type: Task Priority: Major - P3
Reporter: Kay Kim (Inactive) Assignee: Kay Kim (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-28822 Improve DBConnectionPool's growth sem... Closed
Participants:
Days since reply: 5 years, 35 weeks, 5 days ago
Epic Link: DOCS: 4.0 Server
Story Points: 0.5

 Description   

Documentation Request Summary:

This change adds new options and behavior for the dbclient pool, I will add a comment about them.

ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
    maxInUseConnsPerHostParameter(ServerParameterSet::getGlobal(),
                                  "connPoolMaxInUseConnsPerHost",
                                  &ConnPoolOptions::maxInUseConnsPerHost);
 
ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
    maxShardedInUseConnsPerHostParameter(ServerParameterSet::getGlobal(),
                                         "connPoolMaxShardedInUseConnsPerHost",
                                         &ConnPoolOptions::maxShardedInUseConnsPerHost);
 
ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
    globalConnPoolIdleTimeoutParameter(ServerParameterSet::getGlobal(),
                                       "globalConnPoolIdleTimeoutMinutes",
                                       &ConnPoolOptions::globalConnPoolIdleTimeout);
 
ExportedServerParameter<int, ServerParameterType::kStartupOnly>  //
    shardedConnPoolIdleTimeoutParameter(ServerParameterSet::getGlobal(),
                                        "shardedConnPoolIdleTimeoutMinutes",
                                        &ConnPoolOptions::shardedConnPoolIdleTimeout);

Scope of changes:

  • parameters
  • 4.0.txt

Impact to other docs outside of this product:

none

MVP:

Resources:

Engineering Ticket Description:

The DBConnectionPool (legacy connection pool that backs dbclient, used for write operations through 3.5.x) has hard-to-understand and occasionally problematic growth semantics.

Our documentation for the knob that controls this pool is correct:

Set the maximum size of the connection pools for outgoing connections to other mongod instances. The size of a pool does not prevent the creation of additional connections, but does prevent a connection pool from retaining connections in excess of the value of connPoolMaxConnsPerHost.

However, it does not explain that connections retained in the pool will remain open in the pool forever, leading to a steady growth in long-lived outgoing server connections, then a plateau once the max pool size is reached, even when there is no load on the system.

We could do the following things to improve DBConnectionPool's semantics and offer more control over its growth:

  • add a "MaxOpenConnections" option that would cap the number of connections that are simultaneously created (not just retained) by DBConnectionPool. For backwards compatibility this would default to infinity.
  • add an "IdleHostTimeoutMS" option, so that DBConnectionPool dumps its connections to a host after this amount of time spent idle, instead of keeping them open forever. For backwards compatibility this would default to "no timeout."
  • strongly suggest that customers with large clusters or many mongos set the connPoolMaxConnsPerHost and connPoolMaxShardedConnsPerHost parameters to some number that is much smaller than the default of 200.


 Comments   
Comment by Githook User [ 08/Jun/18 ]

Author:

{'username': 'kay-kim', 'name': 'kay', 'email': 'kay.kim@10gen.com'}

Message: DOCS-11281: connection pool params
Branch: v3.6
https://github.com/mongodb/docs/commit/93f880d5d073b93d18d8b280e1963488418b02a7

Comment by Githook User [ 08/Jun/18 ]

Author:

{'username': 'kay-kim', 'name': 'kay', 'email': 'kay.kim@10gen.com'}

Message: DOCS-11281: connection pool params
Branch: master
https://github.com/mongodb/docs/commit/c3ba1f066e94ef6e866c859179605f57b25d70f5

Generated at Thu Feb 08 08:02:29 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.