[SERVER-33972] Bind jstestfuzz version to server version Created: 19/Mar/18 Updated: 06/Dec/22 |
|
| Status: | Backlog |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Robert Guo (Inactive) | Assignee: | Backlog - Server Tooling and Methods (STM) (Inactive) |
| Resolution: | Unresolved | Votes: | 0 |
| Labels: | stm | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Assigned Teams: |
Server Tooling & Methods
|
| Participants: | |
| Story Points: | 2 |
| Description |
|
As the fuzzer becomes more dependent on code in the server repo, like the rollback fixture and the impending preamble hooks, it would be nice to not have to make a change in the server, wait a while, change the fuzzer, wait a bit more and go back to the server to remove code left in place to support the old fuzzer behavior. We should be able to do by making jstestfuzz a module in evergreen.yml |
| Comments |
| Comment by Steven Vannelli [ 10/May/22 ] |
|
Moving this ticket to the Backlog and removing the "Backlog" fixVersion as per our latest policy for using fixVersions. |
| Comment by Max Hirschhorn [ 02/May/19 ] |
|
Every time I come back to think about this ticket, I'm skeptical about doing it. Now that EVG-727 is resolved, having the 10gen/jstestfuzz repository as a module would mean that every mongodb/mongo commit has the "correct" version of the rollback_test.js fixture. There wouldn't be any rough edges around stepback. I acknowledge the dance required to be able to do The only end state I really care about is that the fuzzer doesn't have separate branches corresponding to each branch of the mongodb/mongo repository. Maintaining several different versions of resmoke.py across all supported branches has proven to be challenging. |
| Comment by Robert Guo (Inactive) [ 04/Apr/18 ] |
|
I'm moving the ticket over to 4.1 desired due to additional discussion that came out of the CR. We can revisit this again after 4.0 |
| Comment by Robert Guo (Inactive) [ 30/Mar/18 ] |
|
This ticket will result in some unintended side effects for changing the fuzzer and running fuzzer tests. The intention of this ticket is to solve the problem of having to go back and forth when making changes to the server that the fuzzer depends on. When making any change that requires coordination between the server and the fuzzer, the existing workflow requires the following steps 1. change the server code on master and keep the old server code This ticket will allow fuzzer changes to follow the same workflow as the enterprise modules and remove the need for steps 2 and 4. The new workflow will look like: This workflow is inherently racy and can result in failures in a small number of commits (usually 1) that start between when the server and the fuzzer code is pushed. A similar problem exists for the enterprise module and comes up infrequently. There is discussion on possible fixes in EVG-727 but no conclusion yet. None of the proposed solutions in EVG-727 so far affects the efficacy of this ticket. A notable caveat to this new workflow is that the changes to server master, backports, and changes to the fuzzer repo need to happen simultaneously. Because the fuzzer repo is not versioned according to the server codebase like other modules. If the backports don't happen immediately, the existing 4 step workflow still needs to be followed. As for unintended consequences of doing the work in this ticket, first is that people can no longer restart failing fuzzer tasks caused by fuzzer bugs (not server bugs) and have them succeed as a result of pulling in an updated fuzzer version. This should be the desirable behavior and is consistent with that of other modules. Another side-effect/benefit is that we can now use evergreen patch-set-module for patch builds to the fuzzer repo instead of making manual changes to use a forked repo to evergreen,yml. This again seems desirable to me. Finally, stepbacks need to be turned off for fuzzer tasks to prevent a server bug from stepping back to previous tasks from using an incompatible/outdated version of the fuzzer. |