-
Type: Improvement
-
Resolution: Duplicate
-
Priority: Major - P3
-
Affects Version/s: 4.2.0, 4.4.0, 5.0.0, 6.0.0
-
Component/s: None
-
None
-
Query Execution
-
Sharding EMEA 2021-12-27, Sharding EMEA 2022-01-10
Following a conversation with max.hirschhorn, it appears that while dropping a sharded collection, it is possible for CRUD or DDL events to interleave with the collection drops from the individual shards, based on their respective clusterTimes:
Shard 1 | Shard 2 | Shard 3 | |
---|---|---|---|
T1 | drop | ||
T2 | insert | ||
T3 | drop | ||
T4 | drop |
In the above scenario, for a single-collection stream, this will cause change streams to break in two significant ways:
- As soon as the first drop is observed, we will invalidate the stream. The insert event is therefore never seen by the client.
- If the client attempts to startAfter the invalidate resume token, then we will attempt to swallow the drops on all shards to prevent an immediate re-invalidation of the stream. However, this mechanism is based on the expectation that the next event we see after the resume point is an invalidating event. In the above case, on shard 2 we will stop swallowing as soon as we see the insert, which means that the subsequent drop on the same shard will re-invalidate the resumed stream.
In order for change stream semantics to remain robust through collection drops, there cannot be any CRUD or DDL events on that collection at or later than the clusterTime of the earliest drop event across all shards. Max tells me that this is already the case for sharded renameCollection, and that it may be possible to apply a similar approach to drop. The individual collection drops performed by the dropDatabase command should similarly obey the above constraints.
- duplicates
-
SERVER-70831 Execute the kDropCollection phase of dropCollection under a RecoverableCriticalSection
- Closed
- is related to
-
SERVER-61026 Invert order of shards receiving drop collection participant command
- Closed
-
SERVER-70831 Execute the kDropCollection phase of dropCollection under a RecoverableCriticalSection
- Closed