[SERVER-78382] Verify whether primary step down would occur when changing secondary tags Created: 23/Jun/23  Updated: 12/Jul/23  Resolved: 12/Jul/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Engineering Test Priority: Major - P3
Reporter: Alek Antoniewicz Assignee: Alek Antoniewicz
Resolution: Done Votes: 0
Labels: repl-shortlist
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
Assigned Teams:
Replication
Sprint: Repl 2023-07-24
Participants:

 Description   

The Atlas team is considering using replication tags in Atlas to label nodes as "operational" or "warming". When a new node is added, the disk is available immediately, but it's slow. Today we warm it slowly and make the node available immediately. In the future, we want to make the node unavailable for secondary reads until the warming is complete and full performance is achieved. A replica set tag would be used when:

  • when customers add a new node on Atlas
  • when a node falls off and a new node is added as a replacement.

The affected node would use a special RS tag to indicate it is warming until it is warmed.

 
According to our Docs, changing a replica set tag can trigger a primary step down. While the operation is relatively rare, this could introduce unnecessary elections.
We would like to verify whether the primary step down is likely to occur in the above scenario.



 Comments   
Comment by Alek Antoniewicz [ 12/Jul/23 ]

Excellent, thank you for confirming matthew.russotto@mongodb.com!
I believe we have all we need. Closing

Comment by Matthew Russotto [ 11/Jul/23 ]

Step down due to reconfiguration occurs when the current primary is not a valid primary in the new configuration. Currently this will happen if the primary is removed or if its priority is set to 0. Reconfiguration resulting in step-down is only allowed when

{force: true}

is set.

It is also possible to get an election due to reconfiguration because of priority takeover (if priorities are changed) or because the change resulted loss of majority of the set (only with

{force: true}

), but these technically happen after the reconfig

Just changing node tags without

{force: true}

should not result in elections.

Comment by Alek Antoniewicz [ 23/Jun/23 ]

CC alan.zheng@mongodb.com judah.schvimer@mongodb.com kevin.pytlar@mongodb.com jianling.liu@mongodb.com 

Generated at Thu Feb 08 06:38:12 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.