[SERVER-55539] Allow $match filter based on mod of timestamp to participate in split for optimization Created: 25/Mar/21  Updated: 27/Oct/23  Resolved: 18/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Justin Seyster Assignee: Backlog - Query Execution
Resolution: Gone away Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-52283 Enable feature flag for Allow $change... Closed
Related
is related to SERVER-56872 Add optimization function to apply $m... Closed
Assigned Teams:
Query Execution
Participants:

 Description   

One optimization goal for change streams is to make it possible for multiple observers of a collection, database, or cluster to operate in parallel by splitting the events from the stream into an arbitrary number of sub-sequences. For example, three clients with change streams, with each one observing every third event but staggered so that, together, they observe every event.

The simplest way to split up a stream into sub-sequences for this purpose would be with a mod operator on the operation time. In order to operate on timestamps, a user will need to write a $expr that converts the timestamp to an integer, performs the $mod operation on it, and compares the result with $eq. Note that the non-aggregation $mod operator cannot operate on timestamps, making it unsuitable for this purpose.

To make sure that this pattern really does divide resources between all the open change streams, the resulting $match needs to get pushed as far to the front of the pipeline as possible, which means the splitMatchByModifiedFields() optimization needs to recognize it as a simple read on a single field.



 Comments   
Comment by Bernard Gorman [ 18/Aug/21 ]

This was done as part of SERVER-56872.

Generated at Thu Feb 08 05:36:45 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.