[SERVER-65937] JS tests tagged with requires_fcv_53 do not run in the implicit multiversion passthroughs on the 6.0 branch Created: 25/Apr/22  Updated: 29/Oct/23  Resolved: 17/May/22

Status: Closed
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: 6.0.0-rc6, 6.1.0-rc0

Type: Bug Priority: Critical - P2
Reporter: Max Hirschhorn Assignee: David Bradford (Inactive)
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
Problem/Incident
Related
related to SERVER-67791 Tests are wrongly being excluded from... Closed
is related to SERVER-65924 Modify resharding_histogram_metrics.j... Closed
is related to SERVER-65941 Investigate adding a multiversion can... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v6.0
Sprint: DAG 2022-05-30
Participants:
Linked BF Score: 174
Story Points: 3

 Description   

jstests/sharding/resharding_histogram_metrics.js is an example of a JS test tagged with requires_fcv_53. It isn't being run as part of the sharding_multiversion Evergreen task despite last-continuous being binVersion 5.3 and latest being 6.0 and the test running with FCV 5.3.

Assuming this is a general problem there appear to be a fair number of JavaScript tests which are incorrectly being skipped from the implicit multiversion testing on the 6.0 branch.

$ git grep requires_fcv_53 origin/v6.0 -- jstests/ | wc -l
77



 Comments   
Comment by Githook User [ 17/May/22 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-65937: Support different fcv tags for lts and continuous multiversion tests

(cherry picked from commit 3b4067a108295b079c3d43027caffe2e2c66d7d0)
Branch: v6.0
https://github.com/mongodb/mongo/commit/5c8894d8e0fb53d556675b3eb7b3b434e2e21448

Comment by Githook User [ 17/May/22 ]

Author:

{'name': 'David Bradford', 'email': 'david.bradford@mongodb.com', 'username': 'dbradf'}

Message: SERVER-65937: Support different fcv tags for lts and continuous multiversion tests
Branch: master
https://github.com/mongodb/mongo/commit/3b4067a108295b079c3d43027caffe2e2c66d7d0

Comment by David Bradford (Inactive) [ 12/May/22 ]

For multiversion tests, we use the `requires_fcv_*` tags to exclude tests that won't run against the "old" version of mongo we are testing against. I think the problem here is that we are excluding the same set of tags for both tests against the last_continuous mongo version and the last_lts version, which seem incorrect.

To make this a little more tangible. Here is the multiversion configuration we use for the v6.0 branch:

$ python buildscripts/resmoke.py multiversion-config
last_versions:
- last_lts
- last_continuous
requires_fcv_tag: requires_fcv_51,requires_fcv_52,requires_fcv_53,requires_fcv_60

So we generate tests against both the last_lts and the last_continuous versions. In this case, that would be 5.0 (last_lts) and 5.3 (last_continuous). But we use the same requires_fcv_tag list for both of them (51, 52, 53, 60). For the last_lts case, I believe this is correct. The 5.0 mongo instances won't be able supports tests with those fcv tags. But for the last_continuous case, I think we should only be excluding the 60 tag, since the test will be running against 5.3 mongo instances that should support all the 5* tags.

It seems like we need to treat the fcv tags for last_lts and last_continuous different.

Generated at Thu Feb 08 06:04:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.