[SERVER-33844] Change Streams - provide details for metadata operations instead of (or in addition to) invalidation Created: 13/Mar/18 Updated: 29/Oct/23 Resolved: 10/Jul/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Aggregation Framework |
| Affects Version/s: | None |
| Fix Version/s: | 4.1.1 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Rodrigo Nascimento | Assignee: | Nicholas Zolnierz |
| Resolution: | Fixed | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||
| Sprint: | Query 2018-07-02, Query 2018-07-16 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Description |
|
When watching a database or entire cluster, provide notification of metadata events like collection drops and renames rather than invalidating the stream. In case of watching a single collection, use the same format to provide details about invalidation (whether it was caused by collection drop, db drop, collection rename, etc). Original Description: Change streams report the namespace for operations such as inserts, updates and deletes. The namespace information would be useful in case of the invalidate event. Here is an example where the namespace would be extremely helpful: mdbsrv-01 Application thread-1 opens a change stream on db1.coll1 and thread-2 opens a change stream on db1.coll3. Everything captured from these collections will be send to mdbsrv-02, db2.do_it_now. If db1.coll1 gets dropped, the invalidate event is triggered and a invalidate document is inserted into mdbsrv-02, db2.do_it_now. The invalidate document only contains the _id and operationType: "invalidate". By looking into db2.do_it_now, how do I know which collection has inserted the invalidate document into the collection? |
| Comments |
| Comment by Asya Kamsky [ 07/Apr/18 ] |
Since in the current version (3.6) you can only watch one collection at a time, the namespace is already available to the consumer of change events. In addition, invalidation is not meant to be interpreted as a "change event" - it is strictly an indication that the stream has been invalidated and cannot be continued transparently. However, since we are planning to allow watching more than one collection in a single change stream in the future, I'm going to convert this ticket to an improvement request to provide notification of metadata events (collection or db drops, collection rename, etc) that on a single collection would be an invalidation. |