[SERVER-31429] Use the last fetched instead of the last applied to evaluate sync sources Created: 05/Oct/17 Updated: 30/Oct/23 Resolved: 16/Apr/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication |
| Affects Version/s: | None |
| Fix Version/s: | 4.7.0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Spencer Brody (Inactive) | Assignee: | Xuerui Fa |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | former-quick-wins, neweng | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Sprint: | Repl 2020-04-06, Repl 2020-04-20 | ||||||||
| Participants: | |||||||||
| Description |
|
Currently we check whether we should stop fetching from our current sync source and re-evaluate our sync source selection every time we fetch a batch from our sync source. Part of the check for whether we should change sync sources includes whether we are currently caught up to our sync source (iff our sync source in turn has no sync source and isn't primary). If the batch we just fetched is the one that makes us caught up to our sync source, now we'll wait for another batch to time out its awaitData timeout before changing sync sources. If we checked whether we should re-evaluate our sync source at the end of applying a batch, we could change sync sources faster in the case that our sync source has become isolated from the primary |
| Comments |
| Comment by Githook User [ 16/Apr/20 ] |
|
Author: {'name': 'Xuerui Fa', 'email': 'xuerui.fa@mongodb.com', 'username': 'XueruiFa'}Message: |
| Comment by Spencer Brody (Inactive) [ 23/Jul/18 ] |
|
Why would it? The behavior change is pretty straightforward and non-controversial. The only potential questions are about the implementation approach in the code (although I believe even those were answered by my previous comments), which we generally don't do design documents for. |
| Comment by Gregory McKeon (Inactive) [ 10/Jul/18 ] |
|
spencer can you clarify why the work here wouldn't need design? |
| Comment by Spencer Brody (Inactive) [ 06/Oct/17 ] |
|
Looks like we already decide what sync source to use based on last fetched, per |
| Comment by Spencer Brody (Inactive) [ 06/Oct/17 ] |
|
There might be an easier way to do this by making both the decision about whether we should re-evaluate sync sources and the decision about what new sync source to choose be based on the last op fetched instead of the last op applied |
| Comment by Spencer Brody (Inactive) [ 06/Oct/17 ] |
|
Yeah, this will probably be tricky to implement with the way the code is currently structured, since it would require passing the OplogQueryMetadata through from the Producer to the Applier, and would require a way for the Applier to signal to the Producer that it needs to choose a new sync source. There's nothing fundamental preventing it, but the current code structure will probably make it infeasible in the near term I imagine. |
| Comment by Eric Milkie [ 06/Oct/17 ] |
|
I'm not sure how this would work – do you have the information necessary to make a sync source determination in the applier thread? The current checking is done in the oplog fetcher thread. |