-
Type: Improvement
-
Resolution: Unresolved
-
Priority: Major - P3
-
None
-
Affects Version/s: 3.7.2
-
Component/s: Testing Infrastructure
-
Server Tooling & Methods
The preamble serves as a gateway between the fuzzer and the testing target by plucking out known issues with precision and minimal impact on test coverage. Over the years it has grown in complexity and it makes more and more sense to move it to the server repo, which would bring the following benefits:
1. The fuzzer generated tests will be smaller and more usable without forcing the user to first scroll though 1400+ lines to get to the generated test.
2. Many blacklists are for a specific subset of MongoDB versions, these now no longer need to check for the version number.
3. The process of removing an item from the blacklist after a bug fix can be streamlined and be done under the same ticket as the server bug fix.
4. Pave the way for testing the preamble in the shell (instead of having to set up a separate environment to mimic that of the shell)
5. More strict clang-format rules
6. Reduce duplication with other JS monkey-patching code in our testing infrastructure
The migration will likely involve the following steps
1. Add preamble.js to each MongoDB branch and clean up preamble to remove things not relevant for a particular branch
2. On the fuzzer side, change the existing preamble to not run if it detects the new preamble has already run
3. Modify resmoke to load the new preamble, backport to all branches.
4. Write up a short wiki page on how to blacklist/unblacklist commands for new failures, and state that the artifacts from the failing task will now need to be used to repro an issue.
5. wait a few days until all fuzzer tasks from prior commits have run on all branches
6. remove the old preamble in the fuzzer repo.