[SERVER-21299] resmoke should default to --shuffle Created: 04/Nov/15  Updated: 06/Dec/17  Resolved: 09/Jun/17

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.5.9

Type: Improvement Priority: Major - P3
Reporter: Mathias Stearn Assignee: May Hoque
Resolution: Done Votes: 0
Labels: neweng, tig-resmoke
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: TIG 2017-06-19
Participants:

 Description   

At least when using -j > 1, since the order of tests is already non-deterministic. This will minimize the impact from starting the slowest tests last because they are at the end of the alphabet. I think the worst offenders are sync2.js and tags.js in jstests/replsets. Starting them earlier can shave several minutes off of a -j20 run of the replica_sets suite.



 Comments   
Comment by May Hoque [ 09/Jun/17 ]

have a nice day

Comment by Githook User [ 09/Jun/17 ]

Author:

{u'username': u'Utagai', u'name': u'may', u'email': u'may.hoque@mongodb.com'}

Message: SERVER-21299 resmoke should default to --shuffle
Branch: master
https://github.com/mongodb/mongo/commit/ea8402f7ffe3f7aa5fb3a6ebfa152900d3bf5796

Comment by Mathias Stearn [ 31/May/17 ]

max.hirschhorn Yes, thanks for the link.

Comment by Max Hirschhorn [ 31/May/17 ]

redbeard0531, could you confirm whether what you're asking for with regard to skipping tests that had previous succeeded is covered by Charlie's feature request in SERVER-21074?

Comment by Mathias Stearn [ 30/May/17 ]

I know I asked for this feature, but I sometimes rely on the semi-deterministic ordering with -j > 1. In particular I sometimes will run a suite with something like resmoke --executor=sharding jstests/sharding/[c-z]*.js where c is the first letter of the first test that failed on the last run. This lets me advance the start point to skip known-good tests.

To be clear, I still think we should default to --shuffle, I just would like a way to opt-in to deterministic ordering. (Or some better solution that will let me iteratively fix N tests in less than O(N^2) time.)

Comment by Spencer Brody (Inactive) [ 04/Nov/15 ]

We should probably also drop the 'z' from 'zBigMapReduce' now

Generated at Thu Feb 08 03:56:55 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.