[SERVER-37408] Add afterClusterTime to initial sync collection scans Created: 01/Oct/18 Updated: 29/Oct/23 Resolved: 06/Nov/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Replication, Storage |
| Affects Version/s: | None |
| Fix Version/s: | 3.6.10, 4.0.5, 4.1.5 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Eric Milkie | Assignee: | Matthew Russotto |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Operating System: | ALL | ||||||||||||||||||||||||
| Backport Requested: |
v4.0, v3.6
|
||||||||||||||||||||||||
| Sprint: | Repl 2018-11-05, Repl 2018-11-19 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Linked BF Score: | 68 | ||||||||||||||||||||||||
| Comments |
| Comment by Githook User [ 28/Nov/18 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@10gen.com', 'username': 'mtrussotto'}Message: (cherry picked from commit cbd0a1a3df662c54da23d5def4ccc10dd1c1f88e) |
| Comment by Githook User [ 28/Nov/18 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@10gen.com', 'username': 'mtrussotto'}Message: (cherry picked from commit cbd0a1a3df662c54da23d5def4ccc10dd1c1f88e) |
| Comment by Matthew Russotto [ 28/Nov/18 ] |
|
This bug has a much smaller window to manifest on 3.6 and 3.4 because the S-lock removed in |
| Comment by Githook User [ 06/Nov/18 ] |
|
Author: {'name': 'Matthew Russotto', 'email': 'matthew.russotto@10gen.com', 'username': 'mtrussotto'}Message: |
| Comment by Eric Milkie [ 29/Oct/18 ] |
|
Correct. OplogFetcher::_makeFindCommandObject() already uses afterOpTime instead of afterClusterTime in 3.6 and 3.4. |
| Comment by Matthew Russotto [ 29/Oct/18 ] |
|
I think for 3.4 and 3.6 (for upgrade/downgrade) we'll need to use afterOpTime instead of afterClusterTime. |
| Comment by Judah Schvimer [ 05/Oct/18 ] |
|
The bug here is that without using afterClusterTime, the initial oplog read for the "start timestamp" could return timestamp 7, even if the storage transaction for the operation and oplog entry at timestamp 6 is not yet storage-committed. If at timestamp 6, document 'a' is removed, the collection clone currently could clone 'a', but never fetch the oplog entry at timestamp 6 and thus never remove it. Adding afterClusterTime guarantees we'll have all oplog entries for data we clone. |
| Comment by Judah Schvimer [ 01/Oct/18 ] |
|
This should be reproducible using a test similar to https://github.com/mongodb/mongo/commit/b64de307169891f859c29f207e712ed0eb3cd2a2 and the hangAfterCollectionInserts failpoint: https://github.com/mongodb/mongo/blob/434f7c8c8e66529fbd47f8f6be21df600701385b/jstests/core/txns/speculative_snapshot_includes_all_writes.js#L45-L49 |