[SERVER-33844] Change Streams - provide details for metadata operations instead of (or in addition to) invalidation Created: 13/Mar/18  Updated: 29/Oct/23  Resolved: 10/Jul/18

Status: Closed
Project: Core Server
Component/s: Aggregation Framework
Affects Version/s: None
Fix Version/s: 4.1.1

Type: Improvement Priority: Major - P3
Reporter: Rodrigo Nascimento Assignee: Nicholas Zolnierz
Resolution: Fixed Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Documented
is documented by DOCS-11903 Docs for SERVER-33844: Change Streams... Closed
Duplicate
Backwards Compatibility: Fully Compatible
Sprint: Query 2018-07-02, Query 2018-07-16
Participants:

 Description   

When watching a database or entire cluster, provide notification of metadata events like collection drops and renames rather than invalidating the stream. In case of watching a single collection, use the same format to provide details about invalidation (whether it was caused by collection drop, db drop, collection rename, etc).

Original Description:

Change streams report the namespace for operations such as inserts, updates and deletes. The namespace information would be useful in case of the invalidate event.

Here is an example where the namespace would be extremely helpful:

mdbsrv-01
db1.coll1
db1.coll2
db1.coll3

Application thread-1 opens a change stream on db1.coll1 and thread-2 opens a change stream on db1.coll3. Everything captured from these collections will be send to mdbsrv-02, db2.do_it_now.

If db1.coll1 gets dropped, the invalidate event is triggered and a invalidate document is inserted into mdbsrv-02, db2.do_it_now. The invalidate document only contains the _id and operationType: "invalidate".

By looking into db2.do_it_now, how do I know which collection has inserted the invalidate document into the collection?



 Comments   
Comment by Asya Kamsky [ 07/Apr/18 ]

namespace information would be useful in case of the invalidate event

Since in the current version (3.6) you can only watch one collection at a time, the namespace is already available to the consumer of change events.

In addition, invalidation is not meant to be interpreted as a "change event" - it is strictly an indication that the stream has been invalidated and cannot be continued transparently.

However, since we are planning to allow watching more than one collection in a single change stream in the future, I'm going to convert this ticket to an improvement request to provide notification of metadata events (collection or db drops, collection rename, etc) that on a single collection would be an invalidation.

Generated at Thu Feb 08 04:34:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.