[SERVER-30589] Don't add stable timestamp candidates when in master slave mode Created: 10/Aug/17  Updated: 30/Oct/23  Resolved: 11/Aug/17

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: None
Fix Version/s: 3.5.12

Type: Bug Priority: Major - P3
Reporter: William Schultz (Inactive) Assignee: William Schultz (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to SERVER-29891 Roll Back to Checkpoint: Call setStab... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: Repl 2017-08-21
Participants:
Linked BF Score: 0

 Description   

When running in Master-Slave mode, the replication "commit point" never advances, since there is no consensus protocol used in this mode. For replication recovery and rollback purposes, in the ReplicationCoordinator we maintain a list of "stableTimestampCandidates" which is updated with a new timestamp every time we update our lastApplied optime. The current "stable" timestamp is calculated as the largest timestamp in this list less than the commit point. We will remove timestamps from this list when they are less than the current stable timestamp. If we add timestamps to this list, but the commit point never advances, then the stableTimestampCandidates list will grow unbounded. This can cause performance issues as this list grows and we keep trying to insert things to it every time an operation comes in. This was causing a test to timeout when we insert a few hundred thousand documents (see linked BF).

To fix this, we should check if we are running in master slave mode and refrain from updating the stable timestamp list if so.



 Comments   
Comment by Githook User [ 11/Aug/17 ]

Author:

{'name': 'William Schultz', 'email': 'william.schultz@mongodb.com'}

Message: SERVER-30589 Don't add stable timestamp candidates when in master slave mode
Branch: master
https://github.com/mongodb/mongo/commit/3a6f7017c2577af96c377f1500998565737952e6

Comment by Githook User [ 10/Aug/17 ]

Author:

{'name': 'William Schultz', 'username': 'will62794', 'email': 'william.schultz@mongodb.com'}

Message: Revert "SERVER-30589 Dont update stable timestamp candidates in master slave mode"

This reverts commit 53db8a7a0cde13aa0c54c6979843faac7fa8332e.
Branch: master
https://github.com/mongodb/mongo/commit/803beb68404bd6b0b678ac698b7f2011a64dab12

Comment by William Schultz (Inactive) [ 10/Aug/17 ]

milkie Yes, it seems this could also be an issue in that case. We should probably discuss, generally, the performance impacts of this list growing very large, and how much we are worried about mitigating those impacts.

Comment by Githook User [ 10/Aug/17 ]

Author:

{'username': 'will62794', 'email': 'william.schultz@mongodb.com', 'name': 'William Schultz'}

Message: SERVER-30589 Dont update stable timestamp candidates in master slave mode
Branch: master
https://github.com/mongodb/mongo/commit/53db8a7a0cde13aa0c54c6979843faac7fa8332e

Comment by Eric Milkie [ 10/Aug/17 ]

Is this a problem for replica sets where the commit level doesn't move?

Generated at Thu Feb 08 04:24:21 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.