Uploaded image for project: 'Core Server'
  1. Core Server
  2. SERVER-21074

Add ability to resmoke.py to pick up where you left off running a suite

    XMLWordPrintableJSON

Details

    • Icon: Improvement Improvement
    • Resolution: Won't Fix
    • Icon: Major - P3 Major - P3
    • None
    • 3.2.0-rc0
    • Testing Infrastructure
    • Server Tooling & Methods

    Description

      When running a large suite locally to test your changes, the following is a common work flow

      python buildscripts/resmoke.py --suites=<suite> -j<N>
      

      You notice a test fails, and you figure out why, fix it, then try again

      python buildscripts/resmoke.py --suites=<suite> -j<N>
      

      Then, after repeating the execution of all the other tests, some other test fails, so you repeat the whole process.

      This all can be very time consuming. In many cases, you don't need to re-run all the tests that succeeded the first time. Either the fix was to the specific test, or for some other reason you are confident that they won't start failing due to your change.

      If there was a report.json file present, resmoke.py could use this file to determine which tests in the suite failed or were skipped last time, and only run those. This behavior should probably be behind some flag, since the user would have to know it was 'safe' not to re-run the tests that previously succeeded. As part of this change, it would probably be useful to generate a report.json file on each invocation of resmoke.py, so the user doesn't need to have the foresight to include this option.

      Attachments

        Activity

          People

            backlog-server-stm Backlog - Server Tooling and Methods (STM) (Inactive)
            charlie.swanson@mongodb.com Charlie Swanson
            Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: