[SERVER-72268] Assertion in CETester::getCE when running CE benchmark Created: 20/Dec/22 Updated: 29/Oct/23 Resolved: 18/Jan/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 6.3.0-rc0 |
| Type: | Bug | Priority: | Major - P3 |
| Reporter: | Anton Korshunov | Assignee: | Alya Berciu |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | M5 | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Query Optimization
|
|||||||||||||||||||||||||
| Backwards Compatibility: | Fully Compatible | |||||||||||||||||||||||||
| Operating System: | ALL | |||||||||||||||||||||||||
| Steps To Reproduce: | In src/mongo/db/query/ce/benchmark_test.cpp look for the benchmarked called BucketLargeNumber10HistogramsLargeConjunctions. Modify it by reducing the number of fields to generate histograms for like so:
Compile and run the benchmark:
|
|||||||||||||||||||||||||
| Sprint: | QO 2023-01-09, QO 2023-01-23 | |||||||||||||||||||||||||
| Participants: |
| Description |
|
Expected std::abs((card) - (memoCE)) <= kMaxCEError (1.32146 <= 0.01) @src/mongo/db/query/ce/test_utils.cpp:132 |
| Comments |
| Comment by Githook User [ 18/Jan/23 ] |
|
Author: {'name': 'Alya Berciu', 'email': 'alya.berciu@mongodb.com', 'username': 'alyacb'}Message: |
| Comment by Timour Katchaounov [ 18/Jan/23 ] |
|
A comment on this sentence "Then, after theĀ MemoSubstitution phase/ during the MemoExploration phase, we substitute it with a SargableNode because we have an index on field "d". Because we also have a histogram on "d", we determine that the selectivity of this predicate is 0.02- giving us a CE of 0.055061." in the comment above. The reason a FilterNode is substituted by a SargableNode in the memo substitution phase is not because there is an index, but solely because the filter is sargable. Indexes come into play later when we do access plan (cost-based optimization) . |
| Comment by Alya Berciu [ 17/Jan/23 ] |
|
I did some more investigating. The estimate does not originate from the GroupNode. In fact, we have a predicate filtering on field "d". Initially this is represented in group 2 as a FilterNode. We start by estimating this heuristically (CE 1.37653). Then, after theĀ MemoSubstitution phase/ during the MemoExploration phase, we substitute it with a SargableNode because we have an index on field "d". Because we also have a histogram on "d", we determine that the selectivity of this predicate is 0.02- giving us a CE of 0.055061. After this, we encounter a logical rewrite called SargableFilterReorder, which replaces the Sargable node with the original Filter node because the optimizer does not plan to use the index (this is further supported by the fact that the child group here, group 1, is a SargableNode encoding other predicates in the query). However, it keeps the old estimate in the memo group (which is probably more accurate because we used histograms). By the time CE testing infrastructure sees memo group 2 (after the call to optimize/MemoExploration phase is done), there is a check which fails because it is expecting the estimate on the memo group to match the estimate we would get if we directly estimated the logical node in that group, since we don't estimate filter nodes in the same way as sargable nodes. I think the easiest fix here is to just update the testing infrastructure to stop checking that memo group logical nodes get the same estimate as the one on their memo groups. |