[SERVER-18839] Define suites for subsets of unittests from each major area of the codebase Created: 05/Jun/15  Updated: 05/Feb/16  Resolved: 22/Jun/15

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 3.1.5

Type: Improvement Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Charlie Swanson
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-18790 Scons should provide a way to build a... Closed
Backwards Compatibility: Fully Compatible
Sprint: Quint Iteration 5
Participants:

 Description   

There is a way to run a subset of the unittests through resmoke.py, but it is not clear or intuitive how to do so. We should define YAML suite definitions for the different areas of the code base. Specifically:

  • replication (build/**/repl/*)
  • auth (build/**/auth/*)
  • sharding (build/**/mongo/s/*)
  • query (build/**/query/*)

Old description:
Right now when I want to run all the replication unit tests, I "scons build/unittests.txt", then I copy all the lines of unittests.txt that correspond to repl tests and paste that into the shell. Then I have to scroll up through the pages of output and visually parse the output looking for any indication that a test fails. This is incredibly involved and error prone. We need some way to run a group of unit tests and see clearly and succinctly if any of them failed.



 Comments   
Comment by Githook User [ 22/Jun/15 ]

Author:

{u'username': u'cswanson310', u'name': u'Charlie Swanson', u'email': u'charlie.swanson@mongodb.com'}

Message: SERVER-18839 Define suites for subsets of unittests from each major area of the codebase
Branch: master
https://github.com/mongodb/mongo/commit/438c619ada3f6bfe62225e82eaa199d77d9cc01c

Comment by J Rassi [ 05/Jun/15 ]

The "query" pattern above looks correct for unit tests associated with the query planner, and yes, I would find this useful.

(Note also that there are unit tests in the exec/ directory for query execution, but I would not expect these to run if I were to execute a suite called "query")

Comment by Kaloian Manassiev [ 05/Jun/15 ]

Confirming for sharding unit tests.

Comment by Charlie Swanson [ 05/Jun/15 ]

kaloian.manassiev, rassi@10gen.com Can you confirm I have the right subset of unittests for sharding and query respectively, and that defining suites for running just these tests would be useful?

Comment by Spencer Brody (Inactive) [ 05/Jun/15 ]

Yep, for repl that's right, for auth its the files under the auth directory, can't speak for other areas

Comment by Charlie Swanson [ 05/Jun/15 ]

Sure! If you can get me an accurate definition of which files go in which suites, I can easily do that. (I just guessed that build/**/repl/* was what you wanted for repl)

Comment by Spencer Brody (Inactive) [ 05/Jun/15 ]

Okay, then can we consider this ticket a request to define appropriate yaml config files for the major components of the system? repl, sharding, auth, query, etc?

Comment by Charlie Swanson [ 05/Jun/15 ]

If I understand what you want to do correctly, there is a way to do this. Currently, you can run all unittests through resmoke.py with

python buildscripts/resmoke.py --suites=unittests

And it will appropriately summarize which ones failed. If you want to run only a subset, you would define a new YAML suite configuration file that would specify which files to include in the suite.

Something like this should do what you want: https://gist.github.com/cswanson310/94280d82e059bfdfeeae
If you save that as buildscripts/resmokeconfig/suites/repl_unittests.yml, you should be able to do

python buildscripts/resmoke.py --suites=repl_unittests

And have it behave as you want.

Generated at Thu Feb 08 03:48:54 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.