[SERVER-55535] Performance tests to exercise change streams optimizations Created: 25/Mar/21 Updated: 29/Oct/23 Resolved: 25/Oct/21 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 5.2.0, 5.0.5, 5.1.1 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Justin Seyster | Assignee: | Justin Seyster |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||||||||||||||||||||||
| Backport Requested: |
v5.1, v5.0
|
||||||||||||||||||||||||||||
| Sprint: | QE 2021-10-18, QE 2021-11-01 | ||||||||||||||||||||||||||||
| Participants: | |||||||||||||||||||||||||||||
| Comments |
| Comment by Bernard Gorman [ 01/Oct/22 ] |
|
Hi ywu@stripe.com, The phrase "change streams are optimized" is referring to the work done to resolve Starting in MongoDB 5.1, if the change stream pipeline contains $match and $project stages, we will push those stages down to execute on the shards, so that only the subset of results that the user requested are returned to mongoS. Additionally, we examine the user's $match stage and attempt to rewrite as much of the filter as possible to apply directly to the oplog scan at the very beginning of the change stream pipeline; this allows us to filter out events before any processing or transformation is applied to them. For selective filters that can be rewritten into the oplog, this can result in a significant performance improvement relative to 5.0 and earlier. Hope this helps to clarify the changes in 5.1! Best regards, |
| Comment by Yang Wu [ 30/Sep/22 ] |
|
Hi, I see on the document that mentions:
Could you share any insights around what were the issue in pre-5.1 and what were the optimizations? I only found this one related to memory usage https://jira.mongodb.org/browse/SERVER-36346 – is there some other perf issues?
Thanks! |
| Comment by Githook User [ 12/Nov/21 ] |
|
Author: {'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}Message: (cherry picked from commit 04c9b53b185df98de8e5dfda57420411e59e9cad) |
| Comment by Githook User [ 04/Nov/21 ] |
|
Author: {'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}Message: (cherry picked from commit 04c9b53b185df98de8e5dfda57420411e59e9cad) |
| Comment by Githook User [ 22/Oct/21 ] |
|
Author: {'name': 'Justin Seyster', 'email': 'justin.seyster@mongodb.com', 'username': 'jseyster'}Message: |