[SERVER-50903] No error when change stream's startAtOperationTime is not in oplog Created: 13/Sep/20  Updated: 27/Oct/23  Resolved: 04/Oct/20

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: 4.4.0
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Oleg Pudeyev (Inactive) Assignee: Bernard Gorman
Resolution: Works as Designed Votes: 0
Labels: qexec-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
related to SERVER-32496 Start tailing from the beginning of c... Backlog
related to SERVER-48523 Unconditionally check the first entry... Closed
Operating System: ALL
Sprint: Query 2020-10-19
Participants:

 Description   

I tried creating a change stream with a start at operation time that is surely not in the oplog (1 second past epoch):

MongoDB Enterprise mongos> db.foo.watch([],{startAtOperationTime:Timestamp(1,0)})

This change stream appears to be accepted by the server without errors.

I expected to receive an error if I requested a start time that was not in the oplog.

SO inquiry: https://stackoverflow.com/questions/63860124/how-to-check-if-theres-enough-oplog-history-for-a-change-stream-start-position/63861498#comment112943453_63861498



 Comments   
Comment by Bernard Gorman [ 04/Oct/20 ]

Hey oleg.pudeyev,

I suspect that this is because you are attempting to run the operation on a new deployment. The purpose of the oplog history check is to ensure that no events can have fallen off the oplog between the specified startAtOperationTime and the timestamp of the first entry currently in the oplog. Therefore, if this is a new deployment and the oplog has not yet rolled over itself for the first time, we allow a change stream to start from any timestamp. Because the oplog has never rolled over, no events can ever have fallen off it; the change stream is therefore guaranteed to return every event that has occurred since startAtOperationTime, even if that timestamp pre-dates the existence of the replica set.

If you insert some documents until the oplog on at least one shard rolls over, you should find that re-running the above request throws a ChangeStreamHistoryLost exception (or ChangeStreamFatalError on some earlier versions). If, after rolling over the oplog, you still don't get an exception, then please let us know - that would indeed be a bug.

Hope this helps!

Best regards,
Bernard.

Comment by Eric Sedor [ 15/Sep/20 ]

Thanks oleg.pudeyev; there would need to be some attention to ensuring a change stream can start at the beginning of the oplog, which is a moving target. This seems related to SERVER-32496, but does not seem like a duplicate.

Generated at Thu Feb 08 05:23:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.