[SERVER-68980] CE unit test infrastructure with in-flight CE recording Created: 19/Aug/22  Updated: 29/Oct/23  Resolved: 30/Aug/22

Status: Closed
Project: Core Server
Component/s: None
Affects Version/s: None
Fix Version/s: 6.2.0-rc0

Type: Task Priority: Major - P3
Reporter: Timour Katchaounov Assignee: Alya Berciu
Resolution: Fixed Votes: 0
Labels: M1
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
is duplicated by SERVER-68979 CE unit test infrastructure with in-f... Closed
Backwards Compatibility: Fully Compatible
Sprint: QO 2022-08-22, QO 2022-09-05
Participants:

 Description   

Implement a unit test infrastructure that allows to capture the cardinality estimate of a tested expression during query optimization.

The current (before this task) way of unit testing CE follows the following 
approach: * get an ABT for a query using translatePipeline

  • create a phaseManager either with MemoSubstitutionPhase or both {MemoSubstitutionPhase, MemoExplorationPhase}
  • run the optimizer on the ABT from step 1 and get an optimized ABT
  • call deriveCE on this optimized ABT
  • there are also some tests calling CE against manually constructed ABTs.

The idea with optimizing either with MemoSubstitutionPhase or both {MemoSubstitutionPhase, MemoExplorationPhase} is to get either "raw" FilterNodes, or get their corresponding SargableNodes (in the second case).

This approach is "unrealistic" because it doesn't test the estimation of ABTs in their actual form as estimated by the optimizer. With this approach comparison expressions are represented either as FilterNode (and PathCompare subtrees), or as nodes where all LogicalMemoDelegator nodes are substituted with the actual nodes they point to.

However, during optimization the structure of the nodes is such that child nodes are often delegator nodes that indirectly reference their children in other memo groups.

 

This task should allow us to unit test expressions in the form used during query optimization. The test infrastructure could intercept the first call to deriveCE() in Memo::estimateCE(), and record it "somewhere" in a way suitable for extraction by the unit test. This "somewhere" could be a map inside a special test-only subclas of the Memo class.



 Comments   
Comment by Githook User [ 30/Aug/22 ]

Author:

{'name': 'Alya Berciu', 'email': 'alya.berciu@mongodb.com', 'username': 'alyacb'}

Message: SERVER-68980 Update CE unit-testing to match optimizer
Branch: master
https://github.com/mongodb/mongo/commit/875211fdaeeae66170bab08b1e5090bb77120686

Generated at Thu Feb 08 06:12:16 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.