Uploaded image for project: 'WiredTiger'
  1. WiredTiger
  2. WT-6981

Add randomness to Python test suite runs

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major - P3
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: WT3.2.2, 4.9.0, 4.4.4
    • Component/s: None
    • Labels:
    • Story Points:
      5
    • Sprint:
      Storage - Ra 2021-01-11, Storage - Ra 2021-01-25

      Description

      There is some use of random functions in the Python test suite to generate somewhat random situations or input when appropriate. However we (purposely) use the same seed to get predictable and reproducible runs each time. This is usually appropriate for a functional test suite. However, we might expose some errors if we were to change the initial seed.

      Investigate having a separate run (like our long run) that generates a random seed on each run that is reported at the beginning of the run, and a way to rerun with a given seed. I think for our regular pull request runs, it's still appropriate to have "predictable" randomness.

      Note that if you look for uses of `suite_random` in the test suite there are only a half dozen. However, one of those uses is in how long lists of scenarios are pruned. Changing which scenarios are run may give us some more test coverage.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              jie.chen Jie Chen
              Reporter:
              donald.anderson Donald Anderson
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: