[SERVER-80157] ExclusionProjectionExecutor forces Document cache to load excluded fields Created: 16/Aug/23 Updated: 29/Oct/23 Resolved: 05/Oct/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.2.0-rc0, 5.0.22, 7.0.3, 4.4.26, 6.0.12 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Ian Boros | Assignee: | Parker Felix |
| Resolution: | Fixed | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||
| Assigned Teams: |
Query Execution
|
||||||||
| Backwards Compatibility: | Fully Compatible | ||||||||
| Operating System: | ALL | ||||||||
| Backport Requested: |
v7.1, v7.0, v6.0, v5.0, v4.4
|
||||||||
| Sprint: | QE 2023-10-02, QE 2023-10-16 | ||||||||
| Participants: | |||||||||
| Description |
|
When an ExclusionProjectionExecutor is used to apply a projection like:
The code walks the input document, and for each field, determines whether to "project" it. Since the code for applying the projection is generalized to work for both inclusion and exclusion projections, the value for the field is always provided. By reading the value field, we load it into the document cache. For exclusion projections, the value of the field doesn't matter, since we're going to exclude it. So the work of loading it into cache is completely wasted. Recently, two fast paths have been added for exclusion projection, which do a direct BSON -> BSON transform. These were added under For example, here is a test script which creates one document with a field blocks which is a 40,000 element array. Then it runs a query to exclude just the blocks field. Since blocks is such a large field, the (wasted) time spent loading it into cache dominates the runtime for 4.4:
If you compare this with a run on 6.2 or later, it is much slower (around 30x on my machine). In short, on affected versions, an exclusion projection requires us to completely copy and shred the fields we don't want to keep. This ticket tracks the work of fixing the problem in the default ExclusionProjectionExecutor path. Whether we also want to backport the new fast paths to older versions is a separate question. |
| Comments |
| Comment by Githook User [ 05/Oct/23 ] |
|
Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'}Message: |
| Comment by Githook User [ 05/Oct/23 ] |
|
Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'}Message: |
| Comment by Githook User [ 04/Oct/23 ] |
|
Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'}Message: |
| Comment by Githook User [ 04/Oct/23 ] |
|
Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'}Message: |
| Comment by Githook User [ 04/Oct/23 ] |
|
Author: {'name': 'Parker Felix', 'email': 'parker.felix@mongodb.com', 'username': 'parker-felix'}Message: |