[SERVER-30035] Allow resmoke to spin up multiple fixtures for diff testing passthroughs Created: 06/Jul/17 Updated: 27/Sep/18 Resolved: 27/Sep/18 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | 3.5.9 |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Robert Guo (Inactive) | Assignee: | DO NOT USE - Backlog - Test Infrastructure Group (TIG) |
| Resolution: | Won't Fix | Votes: | 0 |
| Labels: | None | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Participants: |
| Description |
|
For suites like the query fuzzer that run against different fixtures, it'd be nice for resmoke to support handing the shell multiple fixtures with some specific configuration. This could also be useful if we want to add diff testing for other suites that use resmoke fixtures. |
| Comments |
| Comment by Max Hirschhorn [ 10/Jul/17 ] |
Won't the query fuzzer eventually be testing different major and minor versions of MongoDB? I think at that point it'll require having different binaries which may be easier to do via MongoRunner. You could try it with resmoke.py as the mongod_executable should make it possible to reference binaries such as "mongod-3.4" since they'll have been added to the PATH. If you decide to go with the resmoke.py route, I'd also recommend rather than having resmoke.py support "multiple fixtures" to instead define a new Fixture subclass that encapsulates other existing Fixture subclasses (e.g. MongoDFixture). The setup() and teardown() interfaces should be fairly trivial to implement with this approach and you'd need only to expose enough information about the encapsulated fixtures for TestCase.configure() to define some properties on the mogno shell's TestData object in order to know the connection strings for all processes that were spawned. |
| Comment by Robert Guo (Inactive) [ 06/Jul/17 ] |
|
Originally I had planned on using MongoRunner, but since the query fuzzer won't need older mongo binaries (just different FCVs), using resmoke fixtures seemed like an interesting alternative. This might be worth doing if we want to leverage it for additional diff testing; e.g. for the core suite, possibly with some overrides. If there's no likely use case outside of the query fuzzer, then we don't really need to do this... |
| Comment by Max Hirschhorn [ 06/Jul/17 ] |
|
robert.guo, is this going to eventually involve having resmoke.py learn to start clusters running older versions of MongoDB (i.e. multiversion testing)? I wonder if it would be easier for the time being to use MongoRunner instead then. |