[SERVER-58997] Latency spikes from mongos Created: 02/Aug/21  Updated: 06/Dec/22

Status: Open
Project: Core Server
Component/s: None
Affects Version/s: 4.2.14, 4.4.8
Fix Version/s: None

Type: Bug Priority: Major - P3
Reporter: George Wangensteen Assignee: Backlog - Service Architecture
Resolution: Unresolved Votes: 1
Labels: servicearch-q1-2022, servicearch-q4-2021, servicearch-wfbf-day
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
Related
Assigned Teams:
Service Arch
Operating System: ALL
Sprint: Service Arch 2021-08-09, Service Arch 2021-08-23
Participants:
Case:
Story Points: 7

 Description   

We've seen several incidents in versions >= 4.2 of frequent (every couple minutes) mongos-to-client latency spikes that last tens of seconds each. These latency spikes appear to originate on mongos (i.e. latency is not often or consistently observable at concurrently between mongos and mongod). The latency spikes are consistently associated with bursts of connection creation between mongos and mongod. We have a workaround that avoids this issue (increasing the minimum size of the ShardingTaskExecutorPool, the connection pool between mongos and the shards), but should find the root cause of this issue.

 

Acceptance criteria for this ticket: find the root cause of the latency spikes, and fix the issue if the fix is trivial; if not file a ticket to fix the issue. 


Generated at Thu Feb 08 05:46:03 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.