[SERVER-32085] $changeStream reports incorrect documentKey for unsharded collections that become sharded Created: 26/Nov/17 Updated: 30/Oct/23 Resolved: 06/Dec/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework, Replication, Sharding |
| Affects Version/s: | 3.6.0-rc5 |
| Fix Version/s: | 3.6.1, 3.7.1 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Bernard Gorman | Assignee: | Bernard Gorman |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Operating System: | ALL | ||||||||||||||||
| Backport Requested: |
v3.6
|
||||||||||||||||
| Sprint: | Query 2017-12-04, Query 2017-12-18 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
Following The primary shard therefore becomes a 'poison pill' for $changeStream from this point onwards:
The existing change_streams_unsharded_becomes_sharded.js test does not exercise this behaviour because it (a) shards the collection on _id, so the documentKey is the same pre- and post-sharding; and (b) does not retrieve any documents from the $changeStream pre-sharding, so the stream is effectively dormant and the documentKey is not cached. The following patch demonstrates the bug:
|
| Comments |
| Comment by Bernard Gorman [ 06/Apr/18 ] |
|
As of |
| Comment by Asya Kamsky [ 02/Jan/18 ] |
|
Agreed that this should be documented clearly. |
| Comment by Ravind Kumar (Inactive) [ 26/Dec/17 ] |
|
alyson.cabral I'd like to open a DOCS ticket against this so we can update the resume token docs to make a note here. Something like, 'if you have a changestream cursor against an unsharded collection that is then sharded, you must open a new change stream cursor. Resume tokens generated by the original cursor are no longer valid post-sharding." Or we can just generally advise to re-open change stream cursors established on an unsharded collection after sharding that collection. |
| Comment by Githook User [ 06/Dec/17 ] |
|
Author: {'name': 'Bernard Gorman', 'username': 'gormanb', 'email': 'bernard.gorman@gmail.com'}Message: (cherry picked from commit a18859168f73428522d4338fee982329d9d431ed) |
| Comment by Githook User [ 06/Dec/17 ] |
|
Author: {'name': 'Bernard Gorman', 'username': 'gormanb', 'email': 'bernard.gorman@gmail.com'}Message: |
| Comment by Andy Schwerin [ 26/Nov/17 ] |
|
I think the behavior is acceptable for now. We'll need to makeore enhancements to the sharded catalog before we can do better. |
| Comment by Bernard Gorman [ 26/Nov/17 ] |
|
On a related note: even when this bug is fixed, it will continue to be the case that resume tokens from operations that were retrieved pre-sharding will be unusable post-sharding. Is this the intended behaviour? |