[SERVER-48523] Unconditionally check the first entry in the oplog when attempting to resume a change stream Created: 01/Jun/20 Updated: 29/Oct/23 Resolved: 27/Aug/20 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | 4.7.0, 4.4.2, 4.2.11, 4.0.22 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Bernard Gorman | Assignee: | Bernard Gorman |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | qexec-team | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||
| Backport Requested: |
v4.4, v4.2, v4.0, v3.6
|
||||||||||||||||||||||||
| Sprint: | Query 2020-08-24, Query 2020-09-07 | ||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||
| Case: | (copied to CRM) | ||||||||||||||||||||||||
| Description |
|
When determining whether a node's oplog has sufficient history to correctly resume a stream, we first try to retrieve the initial result from the resumed stream. This is because the result may be enough for us to determine that the stream is resumable, without doing any additional work. If the result does not conclusively prove that the stream is resumable, either because it is EOF or chronologically after the resume point, then we must perform a separate, internal query on the oplog via _assertOplogHasEnoughHistory to determine whether its first entry is earlier than the resume point. However, if there are no matching entries in the oplog and we supply a resume point that has already fallen off, our attempt to retrieve the first entry will scan through the entire oplog until it reaches EOF, only to then call _assertOplogHasEnoughHistory and throw an exception. If the oplog is large, then this pointless scan can be quite disruptive. We should instead unconditionally call _assertOplogHasEnoughHistory to check the oplog's history before attempting to pull the first result from the stream. |
| Comments |
| Comment by Githook User [ 21/Oct/20 ] |
|
Author: {'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}Message: (cherry picked from commit 694ed4153b9d5424b5d169fea5c68f99d4dfb45a) |
| Comment by Githook User [ 11/Oct/20 ] |
|
Author: {'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}Message: (cherry picked from commit 694ed4153b9d5424b5d169fea5c68f99d4dfb45a) |
| Comment by Githook User [ 11/Sep/20 ] |
|
Author: {'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}Message: (cherry picked from commit 694ed4153b9d5424b5d169fea5c68f99d4dfb45a) |
| Comment by Githook User [ 27/Aug/20 ] |
|
Author: {'name': 'Bernard Gorman', 'email': 'bernard.gorman@gmail.com', 'username': 'gormanb'}Message: |