Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-79885

Oplog fetching getMore should not set null lastKnownCommittedOpTime if it is not using exhaust cursors

    • Type: Icon: Bug Bug
    • Resolution: Fixed
    • Priority: Icon: Major - P3 Major - P3
    • 7.1.0-rc0, 7.0.1, 6.0.10, 5.0.21, 4.4.25
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Fully Compatible
    • ALL
    • v7.0, v6.0, v5.0, v4.4
    • Repl 2023-08-21
    • 70

      SERVER-78813 fixed a commit point propagation issue for exhaust oplog fetching cursors when the lastKnownCommittedOpTime is null (i.e. timestamp 0, 0). The way it works now after that fix is that the sync source would consider null lastKnownCommittedOpTime as the smallest opTime and would always (when the sync source has a non-null lastCommitted) trigger empty batches to expedite commit point propagation.

      However, this is not desirable for non-exhaust cursors especially when the syncing node isn't able to advance its commit point after receiving oplog batches (e.g. during initial sync as we don't set lastCommitted until after initial sync finishes). So this means that the oplog getMore sent by the syncing node would always have a null lastKnownCommittedOpTime, triggering empty batches unnecessarily every single time towards the end of initial sync (after the initial syncing node catches up).

      4.4+ by default should use exhaust cursors, so this by default isn't an issue. But exhaust cursor could be turned off manually via oplogFetcherUsesExhaust. And then it would become an issue. Additionally, 4.4 in FCV 4.2 will also by default use non-exhaust cursors. So fixing this will help backporting SERVER-78813 to 4.4

            lingzhi.deng@mongodb.com Lingzhi Deng
            lingzhi.deng@mongodb.com Lingzhi Deng
            0 Vote for this issue
            6 Start watching this issue