[SERVER-62656] Decouple mongos-to-config-server connection pool sizing from mongos-to-shard-server connection pool sizing Created: 14/Jan/22  Updated: 29/Oct/23  Resolved: 25/Apr/22

Status: Closed
Project: Core Server
Component/s: Sharding
Affects Version/s: None
Fix Version/s: 6.0.0-rc3, 5.0.10, 4.4.16, 4.2.22, 6.1.0-rc0

Type: Improvement Priority: Major - P3
Reporter: George Wangensteen Assignee: Billy Donahue
Resolution: Fixed Votes: 0
Labels: servicearch-q1-2022
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Documented
is documented by DOCS-15280 Investigate changes in SERVER-62656: ... Closed
Related
Backwards Compatibility: Fully Compatible
Backport Requested:
v6.0, v5.0, v4.4, v4.2
Sprint: Service Arch 2022-04-04, Service Arch 2022-04-18, Service Arch 2022-05-02
Participants:
Linked BF Score: 148
Story Points: 4

 Description   

The ShardingTaskExecutorPool{Min, Max}Size server parameters control the maximum and minimum sizes of connection pools each TaskExecutor can maintain to any mongod. (Most CRUD/agg operations will use these connection pools to reach mongod from mongos). 

The STAR team did some research and determined that different values for these parameters may have performance implications, and came up with tuning recommendations for these parameters for different sized deployments. However, in clusters where CSRS are on differently-sized machines than shards, we would want the size of the connection pools controlled by this parameter to differ depending on if the targeted mongod is a configsvr or shardsvr. 

This is not possible with the current implementation of the parameter, which controls CP sizing for all mongods. We should decouple this to allow for different CP sizing depending  on if the target is a shardserver or configserver.

 Should be doable by giving the ShardingTaskExecutorPoolController a view into the cluster topology (maybe via RSM?) or some info on it so it can determine if a host is a config server or shard server.  



 Comments   
Comment by Githook User [ 13/Jul/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 optional minmax for CSRS pools

(cherry picked from commit b0b3890bdfc47bfd5a3cbc6c3f4d9b53c87441c4)

xcode deduction guide workaround

(cherry picked from commit f96f35813423b88b2b0ce9af8d06d067891c9c39)

Adapt to v4.2 (no LOGV2, no C++17)
Branch: v4.2
https://github.com/mongodb/mongo/commit/8b895dbbdf2b5b4cd1a584db909665a9a720a3fd

Comment by Githook User [ 07/Jul/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 xcode deduction guide workaround
Branch: v5.0
https://github.com/mongodb/mongo/commit/eb3fb4bc1c0c12ae6ab31d0fed21075647e3dee7

Comment by Githook User [ 30/Jun/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 xcode deduction guide workaround
Branch: v4.4
https://github.com/mongodb/mongo/commit/f96f35813423b88b2b0ce9af8d06d067891c9c39

Comment by Githook User [ 28/Jun/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 optional minmax for CSRS pools

(cherry picked from commit b0b3890bdfc47bfd5a3cbc6c3f4d9b53c87441c4)
Branch: v4.4
https://github.com/mongodb/mongo/commit/14476629f4af6e786550bf842277282d6cad31eb

Comment by Githook User [ 28/Jun/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 explicit dependency of mongocryptd on network_interface_factory

(cherry picked from commit bf31e4b84af51ab25afcfa4083c48b68e6d7333c)
Branch: v4.4
https://github.com/10gen/mongo-enterprise-modules/commit/c4235f1cf96d799a35d0eb7ab98bede3ffe859b0

Comment by Githook User [ 21/Jun/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 optional minmax for CSRS pools

(cherry picked from commit b0b3890bdfc47bfd5a3cbc6c3f4d9b53c87441c4)
Branch: v5.0
https://github.com/mongodb/mongo/commit/6f957231d2090ccf9291acf23b581cd606ab60f6

Comment by Githook User [ 21/Jun/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 explicit dependency of mongocryptd on network_interface_factory

(cherry picked from commit bf31e4b84af51ab25afcfa4083c48b68e6d7333c)
Branch: v5.0
https://github.com/10gen/mongo-enterprise-modules/commit/c8b7d03889273ef310a1ef984b21e9dc95d34c34

Comment by Lauren Lewis (Inactive) [ 14/Jun/22 ]

Requesting backports for 5.3, 5.0, 4.4 & 4.2 as per discussion in today's OR Initiative sync.
cc: blake.oler@mongodb.com shameek.ray@mongodb.com

Comment by Githook User [ 26/Apr/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 optional minmax for CSRS pools

(cherry picked from commit b0b3890bdfc47bfd5a3cbc6c3f4d9b53c87441c4)
Branch: v6.0
https://github.com/mongodb/mongo/commit/75cf260b4e7e19aa98acbfad7e610e82df9fd824

Comment by Githook User [ 25/Apr/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 optional minmax for CSRS pools
Branch: master
https://github.com/mongodb/mongo/commit/b0b3890bdfc47bfd5a3cbc6c3f4d9b53c87441c4

Comment by Githook User [ 05/Apr/22 ]

Author:

{'name': 'Billy Donahue', 'email': 'billy.donahue@mongodb.com', 'username': 'BillyDonahue'}

Message: SERVER-62656 explicit dependency of mongocryptd on network_interface_factory
Branch: master
https://github.com/10gen/mongo-enterprise-modules/commit/bf31e4b84af51ab25afcfa4083c48b68e6d7333c

Comment by Billy Donahue [ 29/Mar/22 ]

draft https://github.com/10gen/mongo/pull/4226
Still figuring out how exactly to tell the pool controller about the hosts configsvr-ness or other special attributes.

Generated at Thu Feb 08 05:55:43 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.