[GODRIVER-2526] Heavy allocations in selectByKind Created: 16/Aug/22  Updated: 28/Oct/23  Resolved: 08/Sep/22

Status: Closed
Project: Go Driver
Component/s: None
Affects Version/s: 1.9.1
Fix Version/s: 1.11.0

Type: Improvement Priority: Unknown
Reporter: Peter Ivanov Assignee: Qingyang Hu
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: PNG File image (6).png    
Issue Links:
Related
related to GODRIVER-2626 PR: further optimize selectByKind for... Closed
related to GODRIVER-2627 PR: optimize latency selector Closed
related to GODRIVER-2628 PR: optimize allocations in selectSer... Closed

 Description   

This is how the top of allocation profile looks like in one of our batch processing services.

We use sharded cluster, pool of mongos services, and drvier 1.9.1.

It seems that, in our case at least, this selection in a no-op (all candidates are Mongos), but the allocation burden is very considerable.



 Comments   
Comment by Githook User [ 07/Sep/22 ]

Author:

{'name': 'Qingyang Hu', 'email': '103950869+qingyang-hu@users.noreply.github.com', 'username': 'qingyang-hu'}

Message: GODRIVER-2526 Optimize description.selectByKind. (#1059)

  • GODRIVER-2526 Optimize description.selectByKind by eliminating struct appending.

Co-authored-by: Benjamin Rewis <32186188+benjirewis@users.noreply.github.com>
Branch: master
https://github.com/mongodb/mongo-go-driver/commit/4c8d1ad208b7e709e74ff64b48431c0601368ba3

Comment by Peter Ivanov [ 29/Aug/22 ]

Hi!

Out setup implies autoscaling of mongos instances according to load, but here's some numbers I can share:

  • around 10 mongo clusters a backend instance talks to
  • at any given time, we expect roughly from 5 to 50 mongos instances for a single cluster
  • mongos instances are somewhat often started and killed, so the list behind a SRV record changes pretty frequently
  • so, you could assume that the backend instance has around 10 clients with 20 mongos instances in each one
Comment by Qingyang Hu [ 25/Aug/22 ]

Hi, petr.ivanov.s@gmail.com, can you please let us know how many mongos instances are running? Maybe the cluster topology as well?

Generated at Thu Feb 08 08:38:50 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.