[SERVER-42732] delayed replica performance downgrade Created: 09/Aug/19  Updated: 06/Dec/22  Resolved: 15/Aug/19

Status: Closed
Project: Core Server
Component/s: Performance, Querying, Replication, Usability
Affects Version/s: 3.6.13
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Markus Kral Assignee: Backlog - Triage Team
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Assigned Teams:
Server Triage
Operating System: ALL
Participants:

 Description   

Hi,

 

beacuse of a mixture of hardware-servers and VMs, we are running a replicaset with 4 active members and 2 arbiters.

 

Host1: Primary + arbiter

Host2: Secondary + arbiter

Host3: Secondary

Host4: Secondary

 

As soon as we configure a secondary to be a delayed and hidden replica, queries using the

changeStream features clogg up the system until its unusable.

Without the delayed replica these queries take < 100ms, with the delayed replica deployed these timings increase significantely up to multiple thousands of ms.

Also the systemload is increasing heavily, from ~0.5 without delayed replica, up to observed 6.

 

The documentation offers the following advice for PSA-Configurations (only if one has only 3 members in the replicaset: master-slave-arbiter):

https://docs.mongodb.com/v3.6/reference/read-concern-majority/#disable-read-concern-majority

 

But this should have no effect in our setup, as we still do have a fully functional PSS replication.

 

After removing the delay and setting the replica to hidden=false the problems vanish and the system is stable and performant again.

 

This is reproducible on our setup.

 

We can



 Comments   
Comment by Danny Hatcher (Inactive) [ 15/Aug/19 ]

Using more than one arbiter in a replica set configuration has the opportunity to cause various issues due to having two nodes not actually copying data. The existence of nodes in your replica set still can effect the definitions of "majority"; in your proposed configuration you have a PSSAAH setup which is vastly different from PSS. I recommend refining your environment to only have one arbiter at most (none is ideal) and ensuring that the hidden node cannot vote.

The SERVER project is for bugs and feature suggestions for the MongoDB server and this seems to be an environmental issue. If you need further assistance troubleshooting, I encourage you to ask our community by posting on the mongodb-user group or on Stack Overflow with the mongodb tag.

Generated at Thu Feb 08 05:01:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.