[SERVER-1424] smoke.py could use some work Created: 15/Jul/10  Updated: 02/Jan/18  Resolved: 08/May/15

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.1.3

Type: Improvement Priority: Critical - P2
Reporter: Richard Kreuter (Inactive) Assignee: Max Hirschhorn
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
depends on SERVER-17947 Invariant failure when running oplogs... Closed
depends on SERVER-18256 mongod crashes with assert when tryin... Closed
is depended on by SERVER-17526 option to allow disabling job object ... Closed
is depended on by SERVER-18389 Post-commit resmoke.py tasks Closed
is depended on by SERVER-18268 Support running sharding and replicat... Closed
Duplicate
is duplicated by SERVER-14870 smoke.py should show mongod startup l... Closed
is duplicated by SERVER-17445 Extend ASAN leak detection to all tes... Closed
is duplicated by SERVER-5736 Shuffle order of tests via smoke.py Closed
Related
related to SERVER-32343 dbtest repl suite restores Replicatio... Closed
related to SERVER-32344 register DropPendingCollectionReaper ... Closed
related to SERVER-18263 apply_ops1.js fails if run alone agai... Closed
related to SERVER-18264 index_many.js fails if run alone agai... Closed
is related to SERVER-10910 Mongobridge does not support SSL Closed
is related to SERVER-11207 smoke.py expand wildcards on Windows Closed
is related to SERVER-4959 Check exit code when killing servers ... Closed
is related to SERVER-10909 Replace jstest calls to MongodRunner Closed
Backwards Compatibility: Fully Compatible
Sprint: Quint Iteration 3.1.2, Quint Iteration 3
Participants:

 Description   

Split out the passthrough tests into separate suites. This allows the generated report.json file to record the success/failure of each individual test.

Support parallel execution of C++ unit tests, dbtests, and jstests: (--jobs=N)

  1. C++ unit tests will "just work"
  2. dbtests require separate dbpaths
  3. jstests should dispatch to independent MongoDB deployments

Support grouping different kinds of tests (e.g. C++ unit tests, dbtests, and jstests) so that they can be run together. This allows for customizability in specifying what tests to execute when changes are made to a particular part of the code.

Support including and excluding jstests from suites by using tags. For example, a "no-sharding" tag that appears in the file would prevent it from being run as a part of the sharding jsCore passthrough.

/**
 * This test should not be run in sharding because <reason>.
 *   @tags: [ no-sharding,  # SERVER-XXX
 *          ]
 */

Support running a set of tests multiple times, or until some test fails. (--repeat=N)
Support randomizing the order in which tests execute. (--shuffle)



 Comments   
Comment by Githook User [ 08/May/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-1424 Run tests in MCI with resmoke.py instead of smoke.py.
Branch: master
https://github.com/mongodb/mongo/commit/e5d8177255d2c7a669b8f96ec8a6fa5c82cf1be5

Comment by Githook User [ 08/May/15 ]

Author:

{u'username': u'visemet', u'name': u'Max Hirschhorn', u'email': u'max.hirschhorn@mongodb.com'}

Message: SERVER-1424 Rewrite smoke.py.

Split out the passthrough tests into separate suites. The MongoDB
deployment is started up by resmoke.py so that we can record the
success/failure of each individual test in MCI.

Added support for parallel execution of tests by dispatching to
multiple MongoDB deployments.

Added support for grouping different kinds of tests (e.g. C++ unit
tests, dbtests, and jstests) so that they can be run together. This
allows for customizability in specifying what tests to execute when
changes are made to a particular part of the code.
Branch: master
https://github.com/mongodb/mongo/commit/424314f65e2e0bd9af8f2962260014d1adc7011b

Comment by Githook User [ 18/Sep/14 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-1424 (re)smoke tag parsing and new test library, v0

Includes:

Comment by Githook User [ 15/Sep/14 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: Revert "SERVER-1424 (re)smoke tag parsing and new test library, v0"

This reverts commit fac5571f67e23a1339e20e0621bab0ba2a6b7163.
Branch: master
https://github.com/mongodb/mongo/commit/4677734e7a065141012c1b63140fd64c8ea3170b

Comment by Githook User [ 15/Sep/14 ]

Author:

{u'username': u'gregstuder', u'name': u'Greg Studer', u'email': u'greg@10gen.com'}

Message: SERVER-1424 (re)smoke tag parsing and new test library, v0

Includes:

Generated at Thu Feb 08 02:56:58 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.