-
Type:
Task
-
Resolution: Fixed
-
Priority:
Major - P3
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Optimization
-
Fully Compatible
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Similar to SERVER-121256, the join cost model is currently overestimating the number of random IOs that single table index scans are performing. We currently invoke mackert lohman with docsOutput. This ignores the fact that for a single index key, all index entries are clustered by RID, which perform sort-sparse IO on the collection. We need to instead estimate the NDV of index keys for the scan. This can be done by estimating the NDV for index keys and dividing by the selectivity of the range.
There are two key technical challenges here:
- Arbitrary index scans may contain multikey fields. Our current NDV estimator assumes all fields are non-multikey. I think that SERVER-122379 should address this challenge.
- Neither JoinCostEstimator nor the JoinCardinalityEstimator does not have access to the single table QSNs which we'll need to estimate the NDV of the index keys. This may require some refactoring.
The other thing this ticket should do is invoke the Yao formula to get the number of distinct pages the fetch will read.
After this ticket, we may require a ticket similar to SERVER-122265 which accounts for the sorted-sparse IO for single table index scans.
- is duplicated by
-
SERVER-122145 [Join Optimization] Use Yao's formula for estimation of random I/O for single table index scan
-
- Closed
-
- is related to
-
SERVER-122379 Extend countNDV method to support multikey, filtered cases
-
- In Code Review
-
-
SERVER-122265 [Join Optimization] Costing of INLJ ignores cost of sorted-sparse I/O to fetch records from collection
-
- In Progress
-
-
SERVER-121256 [Join Optimization] INLJ costing - don't consider every probed document as a random IO
-
- Closed
-
- related to
-
SERVER-122381 Investigate plan quality/stability for TPC-H Q14
-
- Closed
-
-
SERVER-123168 [Join Optimization] Code cleanup: Don't pass SamplingEstimators to JoinCardinalityEstimator
-
- Open
-