[SERVER-38928] background dbhash test hook incorrectly relies on a primary's lastApplied being >= its secondary's lastApplied Created: 10/Jan/19 Updated: 29/Oct/23 Resolved: 14/Jan/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Storage |
| Affects Version/s: | None |
| Fix Version/s: | 4.0.6, 4.1.7 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Daniel Gottlieb (Inactive) | Assignee: | Daniel Gottlieb (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||||
| Backport Requested: |
v4.0
|
||||||||||||||||||||||||||
| Steps To Reproduce: |
|
||||||||||||||||||||||||||
| Sprint: | Storage NYC 2019-01-14 | ||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||
| Linked BF Score: | 0 | ||||||||||||||||||||||||||
| Description |
|
The run_check_repl_dbhash_background.js test relies on a primary's lastApplied value being greater than or equal to its secondaries. We can observe whether this property is true by:
In the absence of elections/rollbacks, if the secondary's operation time is greater than the primary, then the property is violated. Because reads don't advance the lastApplied and are concurrent with writers, it's possible for a writer to pause before hitting this statement in an onCommit hook. Inside this window, the write is committed to the storage engine and thus visible in the oplog. A secondary can observe/process/commit the oplog entries, then update its own lastApplied. run_check_repl_dbhash_background.js expects the primary's operation time to be greater than its secondaries. It uses this property to choose an atClusterTime that is expected to be within the history of all nodes. This fails on tests that turn off majority reads as they will advance their history in step with lastApplied (some caveats apply). Notably, this test does not fail when majority reads are on. This is because those configurations use a single voting primary. In those cases the majority commit point (and thus the history secondaries will keep) is in step with the primary's lastApplied. This implies another set of dependencies which may not be getting tested thoroughly (though the window is much narrower). Specifically, a secondary may respond to majority reads with more up to date documents than a primary. |
| Comments |
| Comment by Githook User [ 14/Jan/19 ] |
|
Author: {'username': 'dgottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'name': 'Daniel Gottlieb'}Message: (cherry picked from commit a9177f25ba33a938b5d1bfcc7cc9c17443cdad45) |
| Comment by Githook User [ 11/Jan/19 ] |
|
Author: {'username': 'dgottlieb', 'email': 'daniel.gottlieb@mongodb.com', 'name': 'Daniel Gottlieb'}Message: |