[SERVER-44977] Allow change stream with updateLookup to run directly against a shard mongoD in a sharded cluster Created: 06/Dec/19 Updated: 29/Oct/23 Resolved: 10/Dec/19 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework, Text Search |
| Affects Version/s: | None |
| Fix Version/s: | 4.2.3, 4.3.3 |
| Type: | Improvement | Priority: | Critical - P2 |
| Reporter: | Bernard Gorman | Assignee: | Bernard Gorman |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||
| Backport Requested: |
v4.2
|
||||||||||||
| Sprint: | Query 2019-12-16 | ||||||||||||
| Participants: | |||||||||||||
| Case: | (copied to CRM) | ||||||||||||
| Description |
|
FTS needs the ability to run an updateLookup change stream directly against a shard mongoD in a cluster. At present, this is not possible, because MongoInterfaceShardServer::attachCursorSourceToPipeline only expects to run in the context of a $lookup, and therefore uasserts if the collection is sharded. When a change stream specifies {fullDocument: "updateLookup"} and we come across an update event, we do the updateLookup here. This calls into MongoInterfaceStandalone::lookupSingleDocument and eventually reaches MongoInterfaceStandalone::makePipeline. When this attempts to call attachCursorSourceToPipeline, though, the fact that this is a shard server becomes relevant. Instead of calling MongoInterfaceStandalone::attachCursorSourceToPipeline, we instead call MongoInterfaceShardServer::attachCursorSourceToPipeline. We only expect this to be called for $lookup, never for a change stream; and so we unconditionally throw if the collection we are attempting to read from is sharded. |
| Comments |
| Comment by Githook User [ 10/Dec/19 ] | ||
|
Author: {'email': 'bernard.gorman@mongodb.com', 'name': 'Bernard Gorman', 'username': 'gormanb'}Message: (cherry picked from commit fa76bfb04a062ce031e4d531b9e50a27e1bc2f76) | ||
| Comment by Githook User [ 10/Dec/19 ] | ||
|
Author: {'name': 'Bernard Gorman', 'username': 'gormanb', 'email': 'bernard.gorman@mongodb.com'}Message: | ||
| Comment by Bernard Gorman [ 07/Dec/19 ] | ||
|
The above is due to the fact that we do not need to do a separate updateLookup for a replacement-style update, since the entire replacement document is already present in the oplog entry. | ||
| Comment by Doug Tarr [ 07/Dec/19 ] | ||
|
More info, this only occurs when we use $set in the update:
but does not occur when we use the replacement style update:
| ||
| Comment by Evan Nixon [ 06/Dec/19 ] | ||
|
It might be helpful to note that there is an existing method MongoInterfaceStandalone::attachCursorSourceToPipelineForLocalRead that we use in |