[COMPASS-7001] Investigate changes in PM-1941: Improved change stream shard-targeting and handling of cluster topology changes Created: 05/Jul/23 Updated: 07/Feb/24 |
|
| Status: | Needs Triage |
| Project: | Compass |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Investigation | Priority: | Major - P3 |
| Reporter: | Backlog - Core Eng Program Management Team | Assignee: | Unassigned |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||
| Epic Link: | COMPASS-7048 | ||||
| Sprint: | Up for triaging | ||||
| Description |
|
Original Downstream Change Summary New parameters added to $changeStream aggregation stage. Description of Linked TicketEpic Summary SummaryChange streams on sharded clusters should return correct events when shards are removed or added. MotivationThis has been a user request and will allow us to offer change streams for more use cases while increasing client-side debugability. There are scenarios that may lead to incorrect events for change stream users on sharded clusters. For example: if a shard gets removed from a cluster before we are finished reading events from it and we resume a stream from a point before the shard was removed, we will silently skip every event that occurred on the removed shard between that time and the point where it was removed. This violates one of change streams’ fundamental guarantees (no skipped events). In a serverless world, where we want to be able to add and remove shards at will, this will become an important issue to fix. Cast of Characters
DocumentationScope Document |
| Comments |
| Comment by Anna Henningsen [ 07/Feb/24 ] |
|
We might want to wait for the outcome of the conversation in https://docs.google.com/document/d/1UqPXCCMJtsWFnNlcTKQE63_Ov5Chi4DQTLayT6Rbbpk/edit?disco=AAABF7QKKi4 to check whether this will actually still be relevant. |
| Comment by Anna Henningsen [ 05/Feb/24 ] |
|
| Comment by PM Bot [ 08/Jan/24 ] |
|
Fix Version updated for upstream PM-1941: |
| Comment by PM Bot [ 19/Jul/23 ] |
|
Fix Version updated for upstream PM-1941: |