[SERVER-5736] Shuffle order of tests via smoke.py Created: 01/May/12  Updated: 14/Apr/16  Resolved: 08/May/15

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Eric Milkie Assignee: Max Hirschhorn
Resolution: Duplicate Votes: 1
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Duplicate
duplicates SERVER-1424 smoke.py could use some work Closed
Backwards Compatibility: Fully Compatible
Sprint: Quint Iteration 3
Participants:

 Description   

We'd like to have smoke.py shuffle the tests in a suite. It should take a seed as a parameter to use for randomizing the shuffle. If a seed isn't provided, it should use the current git hash. This way, each build that buildbot runs has a different shuffle for running tests within a suite, but it is still easily reproducible.



 Comments   
Comment by Max Hirschhorn [ 08/May/15 ]

This issue was resolved during the rewrite of smoke.py as part of SERVER-1424. The seed defaults to the current time rather than the githash, but gets logged prior to the test execution for reproducibility.

Comment by Daniel Crosta [ 29/May/12 ]

Can you set a fix version on this and reassign to me?

Comment by Eric Milkie [ 01/May/12 ]

The second half of this work will be to modify master.cfg to pass in a seed for the Nightlies and 2.0's so that they shuffle every night. We typically don't have a new git hash for the 2.0 builds every night so we'll need to come up with a different seed via buildbot master.

Comment by Daniel Crosta [ 01/May/12 ]

Moved from BUILDBOT to SERVER.

Generated at Thu Feb 08 03:09:44 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.