-
Type:
New Feature
-
Resolution: Unresolved
-
Priority:
Major - P3
-
None
-
Affects Version/s: None
-
Component/s: Query Execution
-
Query Execution
-
None
-
None
-
None
-
None
-
None
-
None
-
None
The smart test selector picks a small set of JS tests for a PR patch and it maps those tests to resmoke suite tasks (similar to burn-in style), and also picks C++ unit-test targets. It uses below signals:
1. Git co-change history. A co-change occurs when two files change in the same commit.
2. Manual C++-to-JS test associations that the owning team maintains in YAML.
3. Golden-output test mappings.
4. A static reverse-import graph for JS libraries.
5. A bounded Bazel rdeps query for C++ unit tests.
The selector output drives burn-in for the selected tests. It supplements the required patch build and does not replace it.
Here are logical pieces/blocks (dependencies in brackets):
1. Co-change graph [none]
2. Manual associations [none]
3. Selector engine [1, 2]
4. BF replay corpus and evaluator [3]
5. Burn-in on selected tests [3]
6. query_selection_gen task generation [5]
7. Evergreen wiring and opt-in alias [6]
Goals
1. Land the POC as production code through a series of small, reviewable PRs (https://app.graphite.com/github/pr/10gen/mongo/64357/SERVER-XXXX-Add-co-change-graph?mode=tour)
2. Measure selector recall against a corpus of build failures (BFs) before rollout.
3. Make the selector opt-in, run like the required patch. It is not part of the commit queue.
Out of scope
1. Replacing the required patch build.
2. Persistent code-to-test associations, except manual associations.
Acceptance criteria
1. BF replay report shows JS test recall at or above the agreed target. The working target is 60%.
2. A patch with the query-required-lite alias runs query_selection_gen. The generated burn-in tasks run only on the allowed variants.
3. Setting enabled: false in etc/query_selection_map.yml stops all generated tasks.
1.
|
Co-change graph | SERVER-138862 |
|
In Progress | Chris Wolff | |
2.
|
Manual associations | SERVER-138863 |
|
Backlog | Unassigned | |
3.
|
Selector engine | SERVER-138864 |
|
Backlog | Unassigned | |
4.
|
BF replay corpus and evaluator | SERVER-138865 |
|
Backlog | Unassigned | |
5.
|
Burn-in on selected tests | SERVER-138866 |
|
Backlog | Unassigned | |
6.
|
query_selection_gen | SERVER-138867 |
|
Backlog | Unassigned | |
7.
|
Evergreen Integration | SERVER-138868 |
|
Backlog | Unassigned |