[SERVER-73649] Improve SBE plan cache BudgetEstimator to incorporate the size of the key Created: 06/Feb/23 Updated: 29/Oct/23 Resolved: 02/Jun/23 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | 7.1.0-rc0 |
| Type: | Task | Priority: | Major - P3 |
| Reporter: | Maddie Zechar | Assignee: | Matt Olma |
| Resolution: | Fixed | Votes: | 0 |
| Labels: | neweng, quick-tech-debt | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||||||
| Assigned Teams: |
Query Optimization
|
||||||||||||||||
| Backwards Compatibility: | Minor Change | ||||||||||||||||
| Sprint: | QO 2023-06-12 | ||||||||||||||||
| Participants: | |||||||||||||||||
| Linked BF Score: | 128 | ||||||||||||||||
| Description |
|
The code for the SBE plan cache's BudgetEstimator currently looks like this:
That is, we are only considering the size of the PlanCacheEntry as counting towards the plan cache's memory budget. However, the keys themselves can get quite large – they encode a query shape which can get large, in addition to a bunch of other flags and index descriminators and such. The fact that we are not considering the size of the key means we are underestimating the memory being used by the SBE plan cache, and potentially using more memory than allowed by the planCacheSize parameter configuration. Original descriptionCurrently, SBE Plan cache budget estimations only track the size of the query plans. To be most accurate, we should also track the size of the hashed key and the underlying query shape (from which the hash is generated). This should be completed before before or in tandem with |
| Comments |
| Comment by Githook User [ 02/Jun/23 ] |
|
Author: {'name': 'Matt Olma', 'email': 'matt.olma@mongodb.com', 'username': 'mattsimply'}Message: |
| Comment by David Storch [ 26/Apr/23 ] |
|
I filed Putting this ticket back into the triage queue. cc charlie.swanson@mongodb.com |
| Comment by Anton Korshunov [ 15/Feb/23 ] |
|
I chatted with david.storch@mongodb.com about this ticket and We also noticed that we keep two copies of the keys inside LRUKeyValue - one in _kvList and one in _kvMap and we'd like to fix that too. |