[SERVER-79755] Investigate lowering to SBE non-distinct_scan lastpoint queries Created: 04/Aug/23 Updated: 29/Oct/23 Resolved: 28/Sep/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.2.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Irina Yatsenko (Inactive) | Assignee: | Irina Yatsenko (Inactive) |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Query Integration
|
| Backwards Compatibility: | Fully Compatible |
| Sprint: | QI 2023-10-02 |
| Participants: |
| Description |
The plan won't have DISTINCT_SCAN because the default index {meta:1, time: 1} doesn't match the query sorting requirements, but it will have a bucket-level $group with $last accumulator, followed by unpack, followed by event-level $group. A pipeline like this should be fully lowerable to SBE but I'm not seeing the bucket-level $group being lowered in my tests. We should investigate this and either keep the whole query in the classic engine or ensure that it gets lowered beyond the unpacking stage. This is likely related to |
| Comments |
| Comment by Githook User [ 28/Sep/23 ] |
|
Author: {'name': 'Irina Yatsenko', 'email': 'irina.yatsenko@mongodb.com', 'username': 'IrinaYatsenko'}Message: |
| Comment by Irina Yatsenko (Inactive) [ 27/Sep/23 ] |
|
While the sort is on time, the lastpoint optimization adds a bucket-level sort prior to unpacking and it gets picked up in pipeline_d (so, as a result, bounded sort isn't considered). There is also another place where a group is being created for lastpoint and I missed it. All this is rather messy already but the extra problem is that the optimization also adds a grouping stage at the bucket level prior to unpacking so the lowered unpack stage finds an SBE object rather than a BSONObject in the bucket slot and blows up. For now I'll do the following: |
| Comment by Irina Yatsenko (Inactive) [ 26/Sep/23 ] |
|
Because the $sort is on the timeField, we will keep blocking these pipelines from lowering to SBE because as part of not supporting potential streaming group and bounded sort optimizations yet. However, I've added "SbeCompatibility::fullyCompatible" flag on the bucket-level group and confirmed that the whole pipeline would be lowered if the guard for $sort is removed. I think we should take this fix even if we won't be lowering these pipelines to SBE just yet to avoid surprises in the future. |
| Comment by Irina Yatsenko (Inactive) [ 07/Sep/23 ] |
|
The bucket-level group is created in createBucketGroupForReorder() and is marked as non-SBE-compatible by default. Would need a similar fix to |