[SERVER-27157] replSetRequestVotes doesn't synchronize between processing and recording request Created: 22/Nov/16  Updated: 05/Apr/17  Resolved: 18/Jan/17

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 3.2.12, 3.4.2, 3.5.2

Type: Bug Priority: Critical - P2
Reporter: Mathias Stearn Assignee: Judah Schvimer
Resolution: Fixed Votes: 0
Labels: None

Issue Links:
Backports
Depends
depends on SERVER-27569 Increase test coverage around durabil... Closed
Related
related to SERVER-27154 replSetRequestVotes command should wa... Closed
is related to SERVER-27155 replSetRequestVotes records dryRun vo... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v3.4, v3.2
Sprint: Repl 2017-01-23
Participants:

 Description   

The command processes the vote request then stores it without any synchronization. This can lead to the following sequence:

  1. Host A requests a vote for term 5 which we grant
  2. Host B requests a vote for term 6 which we grant
  3. Record that we voted for host B in term 6
  4. Record that we voted for host A in term 5

Since we record votes using overwrite updates, the term 5 vote replaces the term 6 vote so we lose the fact that we already voted in term 6. Since we only read that document at startup, this can only result in conflicting votes if a shutdown occurs before we vote again. Unlike SERVER-27154 this can still happen with a clean shutdown.



 Comments   
Comment by Judah Schvimer [ 30/Dec/16 ]

We will add this synchronization by adding a predicate to the document update to only update the document if the new term is strictly greater than the old term.

Comment by Spencer T Brody [ 04/Jan/17 ]

I don't think its safe to change the behavior here without first adding test coverage that we actually use the last vote document correctly, otherwise we'll have no way of knowing if we've broken it. Thus I am marking SERVER-27569 as a dependency

Comment by Githook User [ 18/Jan/17 ]

Author:

{u'username': u'judahschvimer', u'name': u'Judah Schvimer', u'email': u'judah@mongodb.com'}

Message: SERVER-27157 replSetRequestVotes should only store votes with higher terms
Branch: master
https://github.com/mongodb/mongo/commit/2369c1863931e5c7f671c33421416c9580f80626

Comment by Githook User [ 20/Jan/17 ]

Author:

{u'username': u'judahschvimer', u'name': u'Judah Schvimer', u'email': u'judah@mongodb.com'}

Message: SERVER-27157 replSetRequestVotes should only store votes with higher terms

(cherry picked from commit 2369c1863931e5c7f671c33421416c9580f80626)
Branch: v3.4
https://github.com/mongodb/mongo/commit/0f4d520a648a11b612c67b1d983f748a10a97fb1

Comment by Githook User [ 24/Jan/17 ]

Author:

{u'username': u'judahschvimer', u'name': u'Judah Schvimer', u'email': u'judah@mongodb.com'}

Message: SERVER-27157 replSetRequestVotes should only store votes with higher terms

(cherry picked from commit 2369c1863931e5c7f671c33421416c9580f80626)
Branch: v3.2
https://github.com/mongodb/mongo/commit/8ccc63824269d37df9d9105d6076492768b4b5b9

Generated at Sun Sep 24 05:05:52 UTC 2017 using JIRA 7.2.10#72012-sha1:2651463a07e52d81c0fcf01da710ca333fcb42bc.