[SERVER-56172] Create "common" yml files for passthrough testing Created: 19/Apr/21  Updated: 29/Oct/23  Resolved: 26/Oct/21

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 5.2.0

Type: Improvement Priority: Major - P3
Reporter: Robert Guo (Inactive) Assignee: Zituo Jin
Resolution: Fixed Votes: 0
Labels: dp-qp-stakeholder-request-2021-04, dp-qp-stakeholder-request-2021-07, tig-hanganalyzer
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Backwards Compatibility: Fully Compatible
Sprint: STM 2021-11-01
Participants:
Story Points: 2

 Description   

Currently, there is no better way to introduce a new passthrough suite than to copy/paste an existing one and see what fails as it burns in. Could we consider creating "common" yml files that contain settings expected to apply in a specific scenario? For example, "kill_primary" suites should all use the "fail_unclean_shutdown_incompatible_commands.js" library and exclude tests that rely on fastcount. It can be easy to miss one of these requirements if you're creating a new passthrough that exhibits multiple behaviors (kill_primary + retryable_writes). It would be simpler to create more complicated suites if we had "building block" yml files. I also spoke to Raiden Worley about this and there's seems to be a longer term discussion in INIT-57 to improve the configuration scheme of tests, but they're not that modular right now. We also spoke about more prolific test-based tagging, which could solve part of this issue.

AC: 

  • any suite that runs the continuous stepdown hook with kill_primary = true --> convert to a matrix suite definition (leverage the code committed here) which will allow you to take a common yaml file and build things on top of it to define the actual suite
  • update the readme to ensure visibility of the new common yaml file 
  • send out an email to database contributors to let them know this is available to be leveraged

Update:

  • We also want to add last-continuous versions of explicit multiversion suites from SERVER-50389


 Comments   
Comment by Githook User [ 26/Oct/21 ]

Author:

{'name': 'Zituo Jin', 'email': 'zituo.jin@mongodb.com', 'username': 'zituo-jin'}

Message: SERVER-56172 Create "common" yml files for passthrough testing
Branch: master
https://github.com/mongodb/mongo/commit/140822cbd72e3cc04331281fdcc339272a76031b

Comment by Raiden Worley (Inactive) [ 26/Aug/21 ]

Now that we've done TIG-2915, Genny's preprocessor is capable of loading arbitrary yaml data from other files and substituting it into a node, also performing parameter substitution. This lets users define "building blocks" at arbitrary levels of granularity, which sounds like what's asked here.

It might be a good idea to extract and generalize Genny's preprocessor so that resmoke can use it as well. Having a common preprocessing syntax between these separate tools could make them more familiar to new users and make switching between them easier.

Generated at Thu Feb 08 05:38:33 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.