Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-55539

Allow $match filter based on mod of timestamp to participate in split for optimization

    • Type: Icon: Improvement Improvement
    • Resolution: Gone away
    • Priority: Icon: Major - P3 Major - P3
    • None
    • Affects Version/s: None
    • Component/s: None
    • Labels:
    • Query Execution

      One optimization goal for change streams is to make it possible for multiple observers of a collection, database, or cluster to operate in parallel by splitting the events from the stream into an arbitrary number of sub-sequences. For example, three clients with change streams, with each one observing every third event but staggered so that, together, they observe every event.

      The simplest way to split up a stream into sub-sequences for this purpose would be with a mod operator on the operation time. In order to operate on timestamps, a user will need to write a $expr that converts the timestamp to an integer, performs the $mod operation on it, and compares the result with $eq. Note that the non-aggregation $mod operator cannot operate on timestamps, making it unsuitable for this purpose.

      To make sure that this pattern really does divide resources between all the open change streams, the resulting $match needs to get pushed as far to the front of the pipeline as possible, which means the splitMatchByModifiedFields() optimization needs to recognize it as a simple read on a single field.

            backlog-query-execution [DO NOT USE] Backlog - Query Execution
            justin.seyster@mongodb.com Justin Seyster
            0 Vote for this issue
            2 Start watching this issue