When running a large suite locally to test your changes, the following is a common work flow

python buildscripts/resmoke.py --suites=<suite> -j<N>

You notice a test fails, and you figure out why, fix it, then try again

python buildscripts/resmoke.py --suites=<suite> -j<N>

Then, after repeating the execution of all the other tests, some other test fails, so you repeat the whole process.

This all can be very time consuming. In many cases, you don't need to re-run all the tests that succeeded the first time. Either the fix was to the specific test, or for some other reason you are confident that they won't start failing due to your change.

If there was a report.json file present, resmoke.py could use this file to determine which tests in the suite failed or were skipped last time, and only run those. This behavior should probably be behind some flag, since the user would have to know it was 'safe' not to re-run the tests that previously succeeded. As part of this change, it would probably be useful to generate a report.json file on each invocation of resmoke.py, so the user doesn't need to have the foresight to include this option.