[SERVER-62981] Make SBE multi-planner's trial period termination condition independent of collection size Created: 25/Jan/22  Updated: 29/Oct/23  Resolved: 27/Jan/22

Status: Closed
Project: Core Server
Component/s: Query Execution, Query Planning
Affects Version/s: None
Fix Version/s: 5.3.0, 5.2.1

Type: Improvement Priority: Major - P3
Reporter: David Storch Assignee: David Storch
Resolution: Fixed Votes: 0
Labels: RDY, query-director-triage
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Related
related to SERVER-63102 Make separate internalQueryPlanEvalua... Closed
related to SERVER-63642 Add serverStatus metrics to measure m... Closed
related to SERVER-63641 Improve SBE multi-planning by choosin... Closed
is related to SERVER-62150 SBE Multiplanning can be slow when su... Open
is related to SERVER-31078 Query planning is very slow during mu... Backlog
Backwards Compatibility: Fully Compatible
Backport Requested:
v5.2
Sprint: QE 2022-02-07
Participants:
Case:
Linked BF Score: 150

 Description   

MongoDB currently uses a runtime plan selection strategy to choose a winning query plan from amongst a set of candidates. This process is called "multi-planning", and it involves partially executing each candidate, and then using the resulting runtime statistics to make a plan choice. The classic execution engine and the SBE execution engine have different implementations of multi-planning. The SBE multi-planner is affected by SERVER-62150, a performance bug which can cause the SBE multi-planner to be much more expensive than the classic multi-planner. This ticket tracks an idea to mitigate SERVER-62150.

The SBE multi-planner inherited some logic from the classic multi-planner where the maximum number of storage reads allowed during the trial period is calculated as either 10,000 or 30% of the collection, whichever is larger. (These constants are configurable as setParameters, but are generally not reconfigured in the field.) This potentially high bound of 30% of the collection size is not overly problematic for the classic multi-planner because of its round-robin behavior – each plan does a small amount of work in a round-robin fashion until a winner is found. This means that with the exception of the scenario described in SERVER-31078, the length of the classic multi-planner trial period should be relatively well bounded.

Not so for the SBE multi-planner. The absence of round-robin plan execution in the SBE multi-planner means that we can easily hit the 30% of the collection size limit even if there is a much better plan available. Therefore, the proposal for this ticket is to introduce a new setParameter which is symmetric with internalQueryPlanEvaluationCollFraction called internalQueryPlanEvaluationCollFractionSbe. The former parameter, internalQueryPlanEvaluationCollFraction, will apply only to the classic multi-planner whereas internalQueryPlanEvaluationCollFractionSbe will apply only to the SBE multi-planner. Importantly, we will choose a default value of 0 for internalQueryPlanEvaluationCollFractionSbe. This means that the length of an individual SBE candidate plan's trial period will be bounded to 10,000 storage cursor reads, which should avoid the worst affects of SERVER-62150.



 Comments   
Comment by Githook User [ 27/Jan/22 ]

Author:

{'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}

Message: SERVER-62981 Make SBE multi-planner trial period length independent of collection size

This patch changes the 'internalQueryPlanEvaluationCollFraction' knob
to apply only to the classic engine. It introduces a separate knob,
'internalQueryPlanEvaluationCollFractionSbe', which applies only to
the SBE engine. The SBE knob has a default of 0, while the classic
engine retains its default of 0.3. This ensures that by default, no
candidate plan will ever do more than 10,000 storage reads during SBE
multi-planning.

(cherry picked from commit f27f088ecf14825a2ae9cedb2c13093287ded84a)
Branch: v5.2
https://github.com/mongodb/mongo/commit/4144aaa465fa4bef36a71caea8e9b6af3b3c7b46

Comment by Githook User [ 27/Jan/22 ]

Author:

{'name': 'David Storch', 'email': 'david.storch@mongodb.com', 'username': 'dstorch'}

Message: SERVER-62981 Make SBE multi-planner trial period length independent of collection size

This patch changes the 'internalQueryPlanEvaluationCollFraction' knob
to apply only to the classic engine. It introduces a separate knob,
'internalQueryPlanEvaluationCollFractionSbe', which applies only to
the SBE engine. The SBE knob has a default of 0, while the classic
engine retains its default of 0.3. This ensures that by default, no
candidate plan will ever do more than 10,000 storage reads during SBE
multi-planning.
Branch: master
https://github.com/mongodb/mongo/commit/f27f088ecf14825a2ae9cedb2c13093287ded84a

Generated at Thu Feb 08 05:56:35 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.