[SERVER-48523] Unconditionally check the first entry in the oplog when attempting to resume a change stream Created: 01/Jun/20  Updated: 29/Oct/23  Resolved: 27/Aug/20

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.7.0, 4.4.2, 4.2.11, 4.0.22

Type: Improvement Priority: Major - P3
Reporter: Bernard Gorman Assignee: Bernard Gorman
Resolution: Fixed Votes: 0
Labels: qexec-team
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-49691 Change streams may be subject to spur... Closed
related to SERVER-49896 Allow aggregate command to fail if mi... Closed
related to SERVER-50580 SBE should obey ASSERT_MIN_TS_HAS_NOT... Closed
is related to SERVER-50903 No error when change stream's startAt... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.4, v4.2, v4.0, v3.6
Sprint: Query 2020-08-24, Query 2020-09-07
Participants:
Case:

 Description   

When determining whether a node's oplog has sufficient history to correctly resume a stream, we first try to retrieve the initial result from the resumed stream. This is because the result may be enough for us to determine that the stream is resumable, without doing any additional work. If the result does not conclusively prove that the stream is resumable, either because it is EOF or chronologically after the resume point, then we must perform a separate, internal query on the oplog via _assertOplogHasEnoughHistory to determine whether its first entry is earlier than the resume point.

However, if there are no matching entries in the oplog and we supply a resume point that has already fallen off, our attempt to retrieve the first entry will scan through the entire oplog until it reaches EOF, only to then call _assertOplogHasEnoughHistory and throw an exception. If the oplog is large, then this pointless scan can be quite disruptive. We should instead unconditionally call _assertOplogHasEnoughHistory to check the oplog's history before attempting to pull the first result from the stream.



 Comments   
Comment by Githook User [ 21/Oct/20 ]

Author:

{'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}

Message: SERVER-48523 Unconditionally check the first entry in the oplog when attempting to resume a change stream

(cherry picked from commit 694ed4153b9d5424b5d169fea5c68f99d4dfb45a)
(cherry picked from commit 9e38cbba7d3efefa59e25cb0411558591036d30b)
Branch: v4.0
https://github.com/mongodb/mongo/commit/f58bdb68407abc68fddc8a91c0d58785423db1b5

Comment by Githook User [ 11/Oct/20 ]

Author:

{'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}

Message: SERVER-48523 Unconditionally check the first entry in the oplog when attempting to resume a change stream

(cherry picked from commit 694ed4153b9d5424b5d169fea5c68f99d4dfb45a)
Branch: v4.2
https://github.com/mongodb/mongo/commit/9e38cbba7d3efefa59e25cb0411558591036d30b

Comment by Githook User [ 11/Sep/20 ]

Author:

{'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}

Message: SERVER-48523 Unconditionally check the first entry in the oplog when attempting to resume a change stream

(cherry picked from commit 694ed4153b9d5424b5d169fea5c68f99d4dfb45a)
Branch: v4.4
https://github.com/mongodb/mongo/commit/87a8196c3319c5ce4c361d5d5196b387fa5410ae

Comment by Githook User [ 27/Aug/20 ]

Author:

{'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}

Message: SERVER-48523 Unconditionally check the first entry in the oplog when attempting to resume a change stream
Branch: master
https://github.com/mongodb/mongo/commit/694ed4153b9d5424b5d169fea5c68f99d4dfb45a

Generated at Thu Feb 08 05:17:22 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.