[SERVER-59290] Re-evaluate sync source after incrementing config version Created: 11/Aug/21  Updated: 29/Oct/23  Resolved: 06/Jan/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.3.0, 5.0.7

Type: Improvement Priority: Major - P3
Reporter: Xuerui Fa Assignee: Gabriel Marks
Resolution: Fixed Votes: 0
Labels: former-quick-wins
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.0
Sprint: Replication 2021-12-13, Replication 2021-12-27, Replication 2022-01-10
Participants:

 Description   

During sync source selection, we attempt to choose a sync source that is not hidden and is a voter if we are a voter. However, it seems possible to do a reconfig to make our sync source non-voting or hidden. It may be an improvement to re-evaluate our sync source if we discover that our sync source has been modified in this way.



 Comments   
Comment by Githook User [ 17/Mar/22 ]

Author:

{'name': 'Gabriel Marks', 'email': 'gabriel.marks@mongodb.com', 'username': 'marksg07'}

Message: SERVER-59290 Improve criteria for shouldChangeSyncSource

(cherry picked from commit a7bf9b5c81e43595d3548b000f5910852eefe6b3)
Branch: v5.0
https://github.com/mongodb/mongo/commit/c0b449903ffccd423737c262d3b5afd73fa36ea8

Comment by Githook User [ 06/Jan/22 ]

Author:

{'name': 'Gabriel Marks', 'email': 'gabriel.marks@mongodb.com', 'username': 'marksg07'}

Message: SERVER-59290 Improve criteria for shouldChangeSyncSource
Branch: master
https://github.com/mongodb/mongo/commit/a7bf9b5c81e43595d3548b000f5910852eefe6b3

Comment by Samyukta Lanka [ 17/Aug/21 ]

The behavior to re-evaluate the sync source as a part of shouldChangeSyncSource existed prior to the Periodically Re-evaluate Sync Sources project, so it is backportable (although the code was refactored a lot as a part of that project, so it might not be simple). I am more of a fan of changing shouldChangeSyncSource for the reasons Judah mentioned above.

Comment by Judah Schvimer [ 17/Aug/21 ]

I think the newlyAdded reconfigs are infrequent enough that I'm not so concerned about them. That said, I think we should differentiate re-evaluating our sync source (shouldChangeSyncSource if I'm correct) from clearing our sync source and choosing a new one (chooseNewSyncSource).

I think we have 2 options:

  1. clear our sync source and choose a new one on every config version change (maybe with an optimization around newlyAdded changes or changes that don't have a meaningful impact on sync source selection).
  2. change shouldChangeSyncSource to return true if one of the criteria is met that would prohibit a sync source from being chosen in chooseNewSyncSource. Ideally these two code paths would consult a shared utility for the acceptability of a given sync source, though that might be difficult in the implementation.

I'm a fan of changing shouldChangeSyncSource personally since I think it's less disruptive and we consult it on every batch in 5.0+. The one caveat is that this solution is not backportable before 5.0 since "Periodically Re-evaluate Sync Sources" was implemented on 5.0. The other change to clear our sync source and choose a new one on config version changes is backportable fairly easily if that's a strong desire. I'm not convinced this is a bad enough behavior though to warrant a backport.

Comment by Xuerui Fa [ 16/Aug/21 ]

We increment the config version as part of the reconfig to remove newlyAdded fields starting from 5.0. Maybe it could be a bit of an optimization to re-evaluate only if the config version was incremented and the new config does not contain the newlyAdded field? CC samy.lanka judah.schvimer matthew.russotto

Comment by Xuerui Fa [ 16/Aug/21 ]

In Triage, we determined that this ticket should implement re-evaluating our sync source every time we increase our config version, since increasing the config version indicates that our sync source may have been changed. Since reconfigs are relatively infrequent, this should not impact performance substantially.

Generated at Thu Feb 08 05:46:52 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.