[SERVER-44977] Allow change stream with updateLookup to run directly against a shard mongoD in a sharded cluster Created: 06/Dec/19  Updated: 29/Oct/23  Resolved: 10/Dec/19

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Text Search
Affects Version/s: None
Fix Version/s: 4.2.3, 4.3.3

Type: Improvement Priority: Critical - P2
Reporter: Bernard Gorman Assignee: Bernard Gorman
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Related
Backwards Compatibility: Fully Compatible
Backport Requested:
v4.2
Sprint: Query 2019-12-16
Participants:
Case:

 Description   

FTS needs the ability to run an updateLookup change stream directly against a shard mongoD in a cluster. At present, this is not possible, because MongoInterfaceShardServer::attachCursorSourceToPipeline only expects to run in the context of a $lookup, and therefore uasserts if the collection is sharded.

When a change stream specifies {fullDocument: "updateLookup"} and we come across an update event, we do the updateLookup here. This calls into MongoInterfaceStandalone::lookupSingleDocument and eventually reaches MongoInterfaceStandalone::makePipeline. When this attempts to call attachCursorSourceToPipeline, though, the fact that this is a shard server becomes relevant. Instead of calling MongoInterfaceStandalone::attachCursorSourceToPipeline, we instead call MongoInterfaceShardServer::attachCursorSourceToPipeline. We only expect this to be called for $lookup, never for a change stream; and so we unconditionally throw if the collection we are attempting to read from is sharded.



 Comments   
Comment by Githook User [ 10/Dec/19 ]

Author:

{'email': 'bernard.gorman@mongodb.com', 'name': 'Bernard Gorman', 'username': 'gormanb'}

Message: SERVER-44977 Allow $changeStream with updateLookup to run directly against a shard mongoD

(cherry picked from commit fa76bfb04a062ce031e4d531b9e50a27e1bc2f76)
Branch: v4.2
https://github.com/mongodb/mongo/commit/6297fc578b17c1782a6ed8e25948d82a98a40fa1

Comment by Githook User [ 10/Dec/19 ]

Author:

{'name': 'Bernard Gorman', 'username': 'gormanb', 'email': 'bernard.gorman@mongodb.com'}

Message: SERVER-44977 Allow $changeStream with updateLookup to run directly against a shard mongoD
Branch: master
https://github.com/mongodb/mongo/commit/fa76bfb04a062ce031e4d531b9e50a27e1bc2f76

Comment by Bernard Gorman [ 07/Dec/19 ]

The above is due to the fact that we do not need to do a separate updateLookup for a replacement-style update, since the entire replacement document is already present in the oplog entry.

Comment by Doug Tarr [ 07/Dec/19 ]

More info, this only occurs when we use $set in the update:

db.fts.update({_id: ObjectId("5deafcba0513ad2c561a71ed")}, { $set: { a: "bye"}})

but does not occur when we use the replacement style update:

db.fts.update({_id: ObjectId("5deafcba0513ad2c561a71ed")},  { a: "bye"})

Comment by Evan Nixon [ 06/Dec/19 ]

It might be helpful to note that there is an existing method MongoInterfaceStandalone::attachCursorSourceToPipelineForLocalRead that we use in 
$_internalSearchBetaIdLookup (an internal stage we use when running FTS queries).

Generated at Thu Feb 08 05:07:31 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.