Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-56872

Add optimization function to apply $match predicates on change streams events directly to oplog entry

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major - P3
    • Resolution: Fixed
    • None
    • 5.1.0-rc0
    • None
    • None
    • Minor Change
    • v5.0
    • Query Execution 2021-05-31, Query Execution 2021-06-14, Query Execution 2021-06-28, Query Execution 2021-07-12, Query Execution 2021-07-26, QE 2021-08-09

    Description

      This optimization function will recognize when a $match applies directly to the output of a $changeStream and construct the additional filters that make it safe to apply $match predicates to the oplog entry without the risk of filtering out transactions or applyOps events.

      There will also be infrastructure to apply the needed rewrites to fields in the $match so that predicates can be applied for oplog entries, but the rewrites themselves will be implemented in future tickets.

      Attachments

        Issue Links

          depends on

          Bug - A problem which impairs or prevents the functions of the product. SERVER-54677 Serialization and deserialization for change streams lookup stages

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-54676 Serialization and deserialization for shard-side change streams stages

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-54937 Remove DocumentSourceChangeStreamCloseCursor dependency on invalidate events

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55534 Store original pipeline in DocumentSourceUpdateOnAddShard for dispatch to new shards

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-56871 Allow $match and $project to swap with $_internalUpdateOnAddShard and $_internalEnsureResumeTokenPresent

          • Major - P3 - Major loss of function.
          • Closed
          is depended on by

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55540 Optimization to apply $match filters on "clusterTime" field before DocumentSourceChangeStreamTransform

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55541 Add "ns" to fields eligible for $match optimization in change streams pipelines

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55542 Add "operationType" to fields eligible for $match optimization in change streams pipelines

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55543 Improve change stream $match pushdown to work for "lsid" and "txnNumber" fields

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55544 Add "to" field to fields eligible for $match optimization in change streams pipelines

          • Major - P3 - Major loss of function.
          • Closed
          is duplicated by

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55543 Improve change stream $match pushdown to work for "lsid" and "txnNumber" fields

          • Major - P3 - Major loss of function.
          • Closed
          is related to

          Bug - A problem which impairs or prevents the functions of the product. SERVER-58857 Change stream accidentally filters out rename event from reshardCollection [5.0 Only]

          • Major - P3 - Major loss of function.
          • Closed
          related to

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55535 Performance tests to exercise change streams optimizations

          • Major - P3 - Major loss of function.
          • Closed

          Improvement - An improvement or enhancement to an existing feature or task. SERVER-55539 Allow $match filter based on mod of timestamp to participate in split for optimization

          • Major - P3 - Major loss of function.
          • Closed

          Activity

            People

              justin.seyster@mongodb.com Justin Seyster
              justin.seyster@mongodb.com Justin Seyster
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: