[SERVER-66369] Implicit multiversion testing generated by burn_in_tests is using binVersion latest only Created: 10/May/22  Updated: 29/Oct/23  Resolved: 29/Jul/22

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 6.1.0-rc0

Type: Bug Priority: Critical - P2
Reporter: Max Hirschhorn Assignee: Mikhail Shchatko
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Depends
is depended on by SERVER-64361 Add the "future git tag multiversion"... Open
Backwards Compatibility: Fully Compatible
Operating System: ALL
Sprint: DAG 2022-05-30, DAG 2022-06-13, DAG 2022-06-27, DAG 2022-07-11, DAG 2022-07-25, DAG 2022-08-08
Participants:
Story Points: 3

 Description   

The modified tests jstests/replsets/sessions_collection_auto_healing.js and jstests/sharding/sessions_collection_auto_healing.js from this patch build were run under the replica_sets_multiversion and sharding_multiversion configurations, respectively. However, the mongod and mongos processes were observed to always be running binVersion latest and therefore weren't providing the additional coverage as the ! Enterprise RHEL 8.0 (implicit multiversion) variant.



 Comments   
Comment by Githook User [ 29/Jul/22 ]

Author:

{'name': 'Mikhail Shchatko', 'email': 'mikhail.shchatko@mongodb.com', 'username': 'MikhailShchatko'}

Message: SERVER-66369 Generate burn_in_tests and burn_in_tags using mongo-task-generator
Branch: master
https://github.com/mongodb/mongo/commit/368542c6aa8a129b461e54582ed0913fc72cdfb8

Comment by Mikhail Shchatko [ 07/Jul/22 ]

With DAG-1879 changes mongo-task-generator will be correctly generating burn_in_tests for multiversion tasks. However for "! Enterprise RHEL 8.0 (implicit multiversion)" buildvariant we need to move burn_in_tags logic to mongo-task-generator, which is DAG-1928.

Comment by David Bradford (Inactive) [ 01/Jun/22 ]

A quick update on this since it has been sitting in in-progress for a while. I started to take a look at this before getting pulled off onto some 6.0 blocker work. It doesn't look like burn_in_tests has ever properly supported multiversion testing. There used to be a specific "burn_in_tests_multiversion" script that was supposed to handle burn_in_tests for multiversion testing, but it looks like that was removed at some point and "support" for running multiversion tests in the normal burn_in_tests was added. However, it doesn't look like it ever worked. It looks like that "support" just ran the multiversion suites in non-multiversion configurations. So it needs to be done properly.

We recently migrated most task generation into a single command for the entire evergreen version. We did not however, migrate burn_in_tests as part of that work. I think it might be time to make that migration. The logic for how to properly generate multiversion tasks lives in that code and the most reliable way to reuse that logic would be to include burn_in generation along with it. I'm going to look into how much effort that would be.

Comment by Max Hirschhorn [ 10/May/22 ]

Bumping this ticket to P2 because it is an issue which can lead Server engineers to incorrectly forget to exclude the test from the implicit multiversion testing via feature flag tagging or etc/backports_required_for_multiversion_tests.yml.

Generated at Thu Feb 08 06:05:14 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.