With yielding, we have to take into account that the oplog collection is a logged table. Since logged tables ignore timestamps, any time we refresh the snapshot, we’ll see the latest data, even if we read at a point-in-time. There are ways to avoid reading new data and losing our capped position during yielding. Remember, we control oplog truncation (so we can’t get “CappedPositionLost” on restore), and oplog entries are immutable. The only change we need to handle is new oplog entries being appended to the end of the oplog. With that in mind, we will take different approaches for sampling vs scanning.

Sampling: random cursors do not support bounds, so when we refresh the snapshot there’s a chance the random cursor can see data that wasn’t available in the initial snapshot. The number of samples is determined before sampling starts, so we maintain a constant bound of samples. The distribution will be skewed as new oplog entries are appended. This approach avoids creating “mega markers” though.

Scanning: we can establish an “end marker” at the start of oplog sampling to define where we should stop, and use a cursor end bound. This can be the last oplog entry in the initial snapshot. We don’t want to sample beyond this point because if new entries arrive faster than we can process them, we could end up sampling indefinitely. Any part of the oplog that remains unsampled would be captured in a single “mega marker”, one marker that covers that entire unsampled range.

With yielding in place, should we still go for lock-free reads? Yes, to avoid blocking at all. Conflicting operations don’t have to wait for database or collection-level locks to be released.