[SERVER-21299] resmoke should default to --shuffle Created: 04/Nov/15 Updated: 06/Dec/17 Resolved: 09/Jun/17 |
|
| Status: | Closed |
| Project: | Core Server |
| Component/s: | Testing Infrastructure |
| Affects Version/s: | None |
| Fix Version/s: | 3.5.9 |
| Type: | Improvement | Priority: | Major - P3 |
| Reporter: | Mathias Stearn | Assignee: | May Hoque |
| Resolution: | Done | Votes: | 0 |
| Labels: | neweng, tig-resmoke | ||
| Remaining Estimate: | Not Specified | ||
| Time Spent: | Not Specified | ||
| Original Estimate: | Not Specified | ||
| Backwards Compatibility: | Fully Compatible |
| Sprint: | TIG 2017-06-19 |
| Participants: |
| Description |
|
At least when using -j > 1, since the order of tests is already non-deterministic. This will minimize the impact from starting the slowest tests last because they are at the end of the alphabet. I think the worst offenders are sync2.js and tags.js in jstests/replsets. Starting them earlier can shave several minutes off of a -j20 run of the replica_sets suite. |
| Comments |
| Comment by May Hoque [ 09/Jun/17 ] |
|
have a nice day |
| Comment by Githook User [ 09/Jun/17 ] |
|
Author: {u'username': u'Utagai', u'name': u'may', u'email': u'may.hoque@mongodb.com'}Message: |
| Comment by Mathias Stearn [ 31/May/17 ] |
|
max.hirschhorn Yes, thanks for the link. |
| Comment by Max Hirschhorn [ 31/May/17 ] |
|
redbeard0531, could you confirm whether what you're asking for with regard to skipping tests that had previous succeeded is covered by Charlie's feature request in |
| Comment by Mathias Stearn [ 30/May/17 ] |
|
I know I asked for this feature, but I sometimes rely on the semi-deterministic ordering with -j > 1. In particular I sometimes will run a suite with something like resmoke --executor=sharding jstests/sharding/[c-z]*.js where c is the first letter of the first test that failed on the last run. This lets me advance the start point to skip known-good tests. To be clear, I still think we should default to --shuffle, I just would like a way to opt-in to deterministic ordering. (Or some better solution that will let me iteratively fix N tests in less than O(N^2) time.) |
| Comment by Spencer Brody (Inactive) [ 04/Nov/15 ] |
|
We should probably also drop the 'z' from 'zBigMapReduce' now |