[SERVER-56872] Add optimization function to apply $match predicates on change streams events directly to oplog entry Created: 12/May/21  Updated: 29/Oct/23  Resolved: 04/Aug/21

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Improvement Priority: Major - P3
Reporter: Justin Seyster Assignee: Justin Seyster
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-54677 Serialization and deserialization for... Closed
depends on SERVER-54676 Serialization and deserialization for... Closed
depends on SERVER-54937 Remove DocumentSourceChangeStreamClos... Closed
depends on SERVER-55534 Store original pipeline in DocumentSo... Closed
depends on SERVER-56871 Allow $match and $project to swap wit... Closed
is depended on by SERVER-55540 Optimization to apply $match filters ... Closed
is depended on by SERVER-55541 Add "ns" to fields eligible for $matc... Closed
is depended on by SERVER-55542 Add "operationType" to fields eligibl... Closed
is depended on by SERVER-55543 Improve change stream $match pushdown... Closed
is depended on by SERVER-55544 Add "to" field to fields eligible for... Closed
Duplicate
is duplicated by SERVER-55543 Improve change stream $match pushdown... Closed
Related
related to SERVER-55535 Performance tests to exercise change ... Closed
related to SERVER-55539 Allow $match filter based on mod of t... Closed
is related to SERVER-58857 Change stream accidentally filters ou... Closed
Backwards Compatibility: Minor Change
Backport Requested:
v5.0
Sprint: Query Execution 2021-05-31, Query Execution 2021-06-14, Query Execution 2021-06-28, Query Execution 2021-07-12, Query Execution 2021-07-26, QE 2021-08-09
Participants:

 Description   

This optimization function will recognize when a $match applies directly to the output of a $changeStream and construct the additional filters that make it safe to apply $match predicates to the oplog entry without the risk of filtering out transactions or applyOps events.

There will also be infrastructure to apply the needed rewrites to fields in the $match so that predicates can be applied for oplog entries, but the rewrites themselves will be implemented in future tickets.



 Comments   
Comment by Vivian Ge (Inactive) [ 06/Oct/21 ]

Updating the fixversion since branching activities occurred yesterday. This ticket will be in rc0 when it’s been triggered. For more active release information, please keep an eye on #server-release. Thank you!

Comment by Githook User [ 06/Aug/21 ]

Author:

{'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}

Message: SERVER-56872 Pass 'allowInexact' to change streams rewrite functions.
Branch: master
https://github.com/mongodb/mongo/commit/b98df93ad2046a5b69f353a7d7b11dca25ef7fbe

Comment by Githook User [ 03/Aug/21 ]

Author:

{'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}

Message: SERVER-56872 Optimize $match in change stream pipeline
Branch: master
https://github.com/mongodb/mongo/commit/c34976fc8f460bdc53440820a5b4d5de035de968

Generated at Thu Feb 08 05:40:25 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.