-
Type: Task
-
Resolution: Done
-
Priority: Major - P3
-
None
-
Affects Version/s: None
-
Component/s: None
-
None
-
Query Execution
-
Fully Compatible
-
QE 2024-07-22, QE 2024-08-05
Framework for testing algorithms against scenarios.
There will be a list of scenario functions, each returning a pair of query stats. The first element of the pair is classic engine, the second is SBE engine.
There will be a list of algorithm functions that take the query stats pairs as inputs. For this milestone, a single placeholder algorithm suffices. These algorithm functions return true if there is a regression and false otherwise.
The scenarios will use a uniform interface to set up and populate their base collections. This interface may start with a very simple data generation method. Task 6 will add more sophistication around e.g. collection size, distribution of values, nested documents, etc.
Note - SPM-3818 may provide detailed statistical information beyond two aggregated query stats buckets. To avoid taking a dependency on that work, we assume for now that the input to the algorithm is bucketed query stats. As progress in 3818 proceeds, we may adapt our input definition as needed.
The placeholder algorithm is very simple and does not depend on any specifics of the input data format.
Result - engineers have a shared testing codebase to collaborate in and can in parallel work on any of the remaining tasks in this milestone.
- is depended on by
-
SERVER-92506 Simulate cases where SBE scans more keys or documents than classic
- Closed
-
SERVER-92507 Simulate cases where SBE is just slower than classic.
- Closed
-
SERVER-92509 Simulate cases where SBE is faster than classic
- Closed
-
SERVER-92510 Simulate SBE plancache flooding
- Closed
-
SERVER-92511 Expand javascript test harness data generation
- Needs Scheduling
- is related to
-
SERVER-92681 Recommend data distributions for base collections in RAMI algo test harness
- Closed