[DOCS-14729] Investigate changes in SERVER-54083: Include scores calculated by PlanRanker in explain "queryPlanner" output and/or "allPlansExecution" Created: 11/Aug/21  Updated: 19/Oct/23

Status: Ready for Work
Project: Documentation
Component/s: Server
Affects Version/s: None
Fix Version/s: 5.1.0-rc0

Type: Task Priority: Major - P3
Reporter: Backlog - Core Eng Program Management Team Assignee: Unassigned
Resolution: Unresolved Votes: 0
Labels: query
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Documented
documents SERVER-54083 Include scores calculated by PlanRank... Closed
Participants:
Days since reply: 2 years, 26 weeks ago

 Description   
Downstream Change Summary

The scores calculated by PlanRanker during the trial period are now outputted alongside each candidate plan (both winning and rejected) in the "allPlansExecution" output (see below for exact location). Note that when "allPlansExecution" verbosity is selected, the score will not be outputted alongside the stats of the fully executed winning plan under the top level of "executionStats", only alongside each plan in the "allPlansExecution" section. For queries where there is only one candidate plan the scores will not be outputted ("allPlansExecution" will be empty in this case anyway).

"allPlansExecution" : [
{
"nReturned" : <int>,
"executionTimeMillisEstimate" : <int>,
"totalKeysExamined" : <int>,
"totalDocsExamined" : <int>,
"score" : <double>,
"executionStages" : {
"stage" : <STAGEA>,
"nReturned" : <int>,
"executionTimeMillisEstimate" : <int>,
...
"inputStage" : {
"stage" : <STAGEB>,
...
"inputStage" :

Unknown macro: { ... }

}
}
},
...
]

Description of Linked Ticket

One important use case for explain is determining why the system chose one candidate plan as the winner and rejected others. This information can currently derived from "allPlansExecution" output, but this requires some internal knowledge of how the PlanRanker's score calculation is implemented. Our internal Query Doctor tool, for example, calculates these scores itself (christopher.harris please correct me if this wrong). There is a risk that Query Doctor could end up diverging from the server's behavior with regards to plan score calculation, thereby misinterpreting the explain output.

We should consider simplifying the situation by simply exposing the plan scores. These could be exposed alongside the "winningPlan" and each "rejectedPlan", as well as alongside each plan in the "allPlansExecution" section. Although these scores are very much subject to change (or even go away entirely) as the internals of the query optimization subsystem change, I don't see a harm in exposing them. This is especially true given that explain's output format is excluded from the stable API, and is subject to change as the implementation details of the query system change.


Generated at Thu Feb 08 08:11:04 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.