[SERVER-31707] Test changeStreams on a sharded collection where the shard doesn't know the collection is sharded Created: 24/Oct/17  Updated: 30/Oct/23  Resolved: 03/Dec/17

Status: Closed
Project: Core Server
Component/s: Aggregation Framework, Replication, Sharding
Affects Version/s: None
Fix Version/s: 3.6.3, 3.7.1

Type: Task Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Nicholas Zolnierz
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Related
related to SERVER-31691 Change streams UUID mismatch uasserts... Closed
related to SERVER-31392 Test changeStreams when an unsharded ... Closed
Backwards Compatibility: Fully Compatible
Backport Requested:
v3.6
Sprint: Query 2017-11-13, Query 2017-12-04, Query 2017-12-18
Participants:

 Description   

The shard collection command makes a best-effort attempt to inform the primary shard that the collection is now sharded, but that message is not required to be delivered. This means that the primary shard owning the previously unsharded collection may not know that the collection is now sharded, even after it is.

We should test that if we get into a situation like that we can still establish a changeStream through a mongos that does know that the collection is sharded.



 Comments   
Comment by Githook User [ 10/Jan/18 ]

Author:

{'email': 'nicholas.zolnierz@mongodb.com', 'name': 'Nick Zolnierz', 'username': 'nzolnierzmdb'}

Message: SERVER-31707: Test changeStreams on a sharded collection where the shard doesn't know the collection is sharded

(cherry picked from commit 09d3d18293c33019814aa4c4ec4a5d8437f06718)
Branch: v3.6
https://github.com/mongodb/mongo/commit/e30ccaa2ec1a68607513e1c2316a374736ec4eba

Comment by Githook User [ 03/Dec/17 ]

Author:

{'username': 'nzolnierzmdb', 'email': 'nicholas.zolnierz@mongodb.com', 'name': 'Nick Zolnierz'}

Message: SERVER-31707: Test changeStreams on a sharded collection where the shard doesn't know the collection is sharded
Branch: master
https://github.com/mongodb/mongo/commit/09d3d18293c33019814aa4c4ec4a5d8437f06718

Comment by Spencer Brody (Inactive) [ 27/Oct/17 ]

I confirmed with charlie.swanson that the mongod decides whether to do the updateLookup locally or defer it to the mongos based on whether the mongos believes the collection to be sharded, so I don't think there will be an issue with both mongos and mongod running the updateLookup.

So I believe this should be no worse than the case where the stream is established when the collection is unsharded and then it later becomes sharded. Still probably worth testing to confirm and understand the current behavior.

Comment by Esha Maharishi (Inactive) [ 24/Oct/17 ]

There's a (maybe smaller) wrinkle - will mongos attach version UNSHARDED to shards it believes don't own any chunks?

If so, if a shard donates its last chunk, I don't think we reset its CollectionMetadata, and even making that shard refresh won't cause its CollectionMetadata to reflect version UNSHARDED.

In this case, mongos may never be able to establish the change streams cursor, because it will keep getting a stale shard version error from that shard.

Comment by Spencer Brody (Inactive) [ 24/Oct/17 ]

I wonder if this could be solved by just actually sending a shard version on the agg we send from mongos to mongod. We originally opt-ed out of the versioning protocol because we need changeStreams to target all shards, not just the shards that have chunks, so we didn't want to hook into the existing mongos targeting/refresh logic. We still need to target all shards, but I wonder if it would actually be harmful to just also do versioning. That would at least ensure that the mongod had the right metadata when the cursor is established. I think the only wrinkle would be making sure the mongos-side code was resilient to receiving a StaleShardVersion error during cursor establishment.

charlie.swanson esha.maharishi

Comment by Spencer Brody (Inactive) [ 24/Oct/17 ]

If both the mongos and the mongod agree that the collection is unsharded then there shouldn't be a problem, it reduces to that case where a stream is established on an unsharded collection that then becomes sharded.

I'm worried about a situation where the mongos knows the collection is sharded but the mongod does not. In that case the mongos might try to run the sharded-collection specific stages, but the mongod will not have included the shard key in the ResumeToken. This seems like it could also be a problem for updateLookup, if both the mongod AND the mongos try to do the lookup.

Generated at Thu Feb 08 04:27:57 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.