[SERVER-78231] Change Stream is not working with on-demand materialized views Created: 20/Jun/23  Updated: 01/Sep/23

Status: Needs Scheduling
Project: Core Server
Component/s: None
Affects Version/s: 5.0.7
Fix Version/s: None

Type: Bug Priority: Minor - P4
Reporter: Kos Chaus Assignee: Katya Kamenieva
Resolution: Unresolved Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
Related
related to SERVER-78232 Change Stream is not working with on-... Closed
Assigned Teams:
Query Execution
Operating System: ALL
Steps To Reproduce:
  1. Create collection C1
  2. Create the on-demand materialized view MV1 on the C1 with all matched
  3. Watch MV1
  4. Insert something in C1
  5.  

Expected:

The change in MV1 is caught by the watcher

Actual:

The change in MV1 is NOT caught by the watcher

Participants:

 Description   

The same code works with a regular Mongo collection but doesn't catch any events happening to the on-demand materialized view.

 

I found the test which checks that the Change Stream is not supported for normal views.

However, I didn't find any limitations/restrictions to using Change Streams with on-demand materialized views.

 



 Comments   
Comment by Katya Kamenieva [ 01/Sep/23 ]

k.chaus@efg.gg could you provide more details for steps 2,3,5.
How your materialized view is created and refreshed, what is the pipeline? What does the spec of the change stream look like?

Comment by Kos Chaus [ 21/Jun/23 ]

Hi chris.kelly@mongodb.com 

Thank you for your quick reply!

Sorry, my initial description had lost p.5 for some reason, where I did refresh my materialized view.

Steps to reproduce:

  1. Create collection C1
  2. Create the on-demand materialized view MV1 on the C1 with all matched
  3. Watch MV1
  4. Insert something in C1
  5. Refresh MV1

Did I get your answer correctly that Views (including materialized views) CANNOT be watched by Change Stream API?

If yes, I would expect to get a clear error when trying to watch materialized views.


In my use case, MV1 has a grouped representation of an underlying collection, so it makes only sense to watch MV1, not C1.

 

 

Comment by Chris Kelly [ 20/Jun/23 ]

Hi k.chaus@efg.gg!

Thanks for concisely summarizing the issue. Some notes:

  • Views (including materialized views) are read-only.
  • For change streams, you should watch the underlying collection(s) the view is based on.
  • SERVER-31720 is on the backlog which would likely cover the use case you're suggesting (Provide a way to get notification when results of a query change), and I would suggest looking there to push on that if it's a feature you feel strongly about.
  • Additionally, one extra note is that you must refresh your materialized view to have any changes in the underlying collection(s) reflected in your view.

This does not appear explicit in the change stream docs, and pointing that out is valuable for us. I'll forward this suggestion so we can get this limit clearly displayed!

Christopher

Generated at Thu Feb 08 06:37:47 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.