[SERVER-79885] Oplog fetching getMore should not set null lastKnownCommittedOpTime if it is not using exhaust cursors Created: 09/Aug/23  Updated: 29/Oct/23  Resolved: 11/Aug/23

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 7.1.0-rc0, 7.0.1, 6.0.10, 5.0.21, 4.4.25

Type: Bug Priority: Major - P3
Reporter: Lingzhi Deng Assignee: Lingzhi Deng
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
is caused by SERVER-78813 Commit point propagation fails indefi... Closed
Related
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v7.0, v6.0, v5.0, v4.4
Sprint: Repl 2023-08-21
Participants:
Linked BF Score: 70

 Description   

SERVER-78813 fixed a commit point propagation issue for exhaust oplog fetching cursors when the lastKnownCommittedOpTime is null (i.e. timestamp 0, 0). The way it works now after that fix is that the sync source would consider null lastKnownCommittedOpTime as the smallest opTime and would always (when the sync source has a non-null lastCommitted) trigger empty batches to expedite commit point propagation.

However, this is not desirable for non-exhaust cursors especially when the syncing node isn't able to advance its commit point after receiving oplog batches (e.g. during initial sync as we don't set lastCommitted until after initial sync finishes). So this means that the oplog getMore sent by the syncing node would always have a null lastKnownCommittedOpTime, triggering empty batches unnecessarily every single time towards the end of initial sync (after the initial syncing node catches up).

4.4+ by default should use exhaust cursors, so this by default isn't an issue. But exhaust cursor could be turned off manually via oplogFetcherUsesExhaust. And then it would become an issue. Additionally, 4.4 in FCV 4.2 will also by default use non-exhaust cursors. So fixing this will help backporting SERVER-78813 to 4.4



 Comments   
Comment by Githook User [ 22/Aug/23 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-79885: Oplog fetching getMore should only set null lastKnownCommittedOpTime for exhaust cursors

(cherry picked from commit eeac78cd8de74ca1cffb18eb4b798b8392df6192)
Branch: v7.0
https://github.com/mongodb/mongo/commit/425a0454d12f2664f9e31002bbe4a386a25345b5

Comment by Githook User [ 22/Aug/23 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-79885: Oplog fetching getMore should only set null lastKnownCommittedOpTime for exhaust cursors

(cherry picked from commit eeac78cd8de74ca1cffb18eb4b798b8392df6192)
Branch: v4.4
https://github.com/mongodb/mongo/commit/d7ef660c7871cd3df8d77d3a55b09863ea3e6b22

Comment by Githook User [ 15/Aug/23 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-79885: Oplog fetching getMore should only set null lastKnownCommittedOpTime for exhaust cursors

(cherry picked from commit eeac78cd8de74ca1cffb18eb4b798b8392df6192)
Branch: v5.0
https://github.com/mongodb/mongo/commit/c8326785cfdcd3497c1cf865e637a6b1cfc629e9

Comment by Githook User [ 14/Aug/23 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-79885: Oplog fetching getMore should only set null lastKnownCommittedOpTime for exhaust cursors

(cherry picked from commit eeac78cd8de74ca1cffb18eb4b798b8392df6192)
Branch: v6.0
https://github.com/mongodb/mongo/commit/5b7055580d9cc3d0870b8de4b3d1bcf553b44146

Comment by Githook User [ 11/Aug/23 ]

Author:

{'name': 'Lingzhi Deng', 'email': 'lingzhi.deng@mongodb.com', 'username': 'ldennis'}

Message: SERVER-79885: Oplog fetching getMore should only set null lastKnownCommittedOpTime for exhaust cursors
Branch: master
https://github.com/mongodb/mongo/commit/eeac78cd8de74ca1cffb18eb4b798b8392df6192

Generated at Thu Feb 08 06:42:10 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.