[SERVER-37408] Add afterClusterTime to initial sync collection scans Created: 01/Oct/18  Updated: 29/Oct/23  Resolved: 06/Nov/18

Status: Closed
Project: Core Server
Component/s: Replication, Storage
Affects Version/s: None
Fix Version/s: 3.6.10, 4.0.5, 4.1.5

Type: Bug Priority: Major - P3
Reporter: Eric Milkie Assignee: Matthew Russotto
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
is depended on by SERVER-34243 listCollections should not require a ... Closed
Related
is related to SERVER-30724 Initial sync might miss ops that were... Closed
is related to SERVER-30927 Use readConcern afterClusterTime for ... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.0, v3.6
Sprint: Repl 2018-11-05, Repl 2018-11-19
Participants:
Linked BF Score: 68

 Comments   
Comment by Githook User [ 28/Nov/18 ]

Author:

{'name': 'Matthew Russotto', 'email': 'matthew.russotto@10gen.com', 'username': 'mtrussotto'}

Message: SERVER-37408 Add afterClusterTime to initial sync collection scans

(cherry picked from commit cbd0a1a3df662c54da23d5def4ccc10dd1c1f88e)
Branch: v3.6
https://github.com/mongodb/mongo/commit/96c21fca9277366ace909a53ae67bba26f1abac2

Comment by Githook User [ 28/Nov/18 ]

Author:

{'name': 'Matthew Russotto', 'email': 'matthew.russotto@10gen.com', 'username': 'mtrussotto'}

Message: SERVER-37408 Add afterClusterTime to initial sync collection scans

(cherry picked from commit cbd0a1a3df662c54da23d5def4ccc10dd1c1f88e)
Branch: v4.0
https://github.com/mongodb/mongo/commit/2fd83edf0919e45cbb8c4dbe9fd84d0284877b58

Comment by Matthew Russotto [ 28/Nov/18 ]

This bug has a much smaller window to manifest on 3.6 and 3.4 because the S-lock removed in SERVER-34243 still exists in those revisions.

Comment by Githook User [ 06/Nov/18 ]

Author:

{'name': 'Matthew Russotto', 'email': 'matthew.russotto@10gen.com', 'username': 'mtrussotto'}

Message: SERVER-37408 Add afterClusterTime to initial sync collection scans
Branch: master
https://github.com/mongodb/mongo/commit/cbd0a1a3df662c54da23d5def4ccc10dd1c1f88e

Comment by Eric Milkie [ 29/Oct/18 ]

Correct. OplogFetcher::_makeFindCommandObject() already uses afterOpTime instead of afterClusterTime in 3.6 and 3.4.

Comment by Matthew Russotto [ 29/Oct/18 ]

I think for 3.4 and 3.6 (for upgrade/downgrade) we'll need to use afterOpTime instead of afterClusterTime.

Comment by Judah Schvimer [ 05/Oct/18 ]

The bug here is that without using afterClusterTime, the initial oplog read for the "start timestamp" could return timestamp 7, even if the storage transaction for the operation and oplog entry at timestamp 6 is not yet storage-committed. If at timestamp 6, document 'a' is removed, the collection clone currently could clone 'a', but never fetch the oplog entry at timestamp 6 and thus never remove it. Adding afterClusterTime guarantees we'll have all oplog entries for data we clone.

Comment by Judah Schvimer [ 01/Oct/18 ]

This should be reproducible using a test similar to https://github.com/mongodb/mongo/commit/b64de307169891f859c29f207e712ed0eb3cd2a2 and the hangAfterCollectionInserts failpoint: https://github.com/mongodb/mongo/blob/434f7c8c8e66529fbd47f8f6be21df600701385b/jstests/core/txns/speculative_snapshot_includes_all_writes.js#L45-L49

Generated at Thu Feb 08 04:45:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.