[SERVER-30794] Make it easy to update jstests that set FCV after a major version release Created: 23/Aug/17  Updated: 06/Dec/22

Status: Backlog
Project: Core Server
Component/s: Testing Infrastructure
Affects Version/s: None
Fix Version/s: None

Type: Improvement Priority: Major - P3
Reporter: Spencer Brody (Inactive) Assignee: Backlog - Replication Team
Resolution: Unresolved Votes: 0
Labels: neweng
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Related
is related to SERVER-27355 Avoid hard-coding 3.2 and 3.4 feature... Closed
Assigned Teams:
Replication
Participants:

 Description   

The shell's test infrastructure currently supports the aliases "latest" and "last-stable" when starting mongodb processes, where those aliases map to the current and previous major mongodb version number, which can be updated in one place when we branch for a new major version. It would be nice to have a similar facility for FCV. Some tests, especially in the multiversion suite, need to explicitly set the FCV to the "previous" version so that can run a mixed-version set, it would be nice to not have to locate and manually update all such tests when the major version changes.



 Comments   
Comment by Max Hirschhorn [ 30/Aug/17 ]

Per a Slack conversation with Spencer, I'm moving this ticket over to the Replication team's backlog since introducing the FeatureCompatibilityVersion.kLatest and FeatureCompatibilityVersion.kLastStable constants is small and they are the ones who added the generic tests around feature compatibility version.

Comment by Max Hirschhorn [ 30/Aug/17 ]

Those tests use the "last-stable" and "latest" labels for picking the mongod versions, but the FCV is hard coded and will need to be updated when we branch for 3.8, would be nice if they could be updated automatically.

Just a couple points of clarification:

  • The concept of "last-stable" isn't updated automatically on master after we branch for a new release. The purpose of SERVER-27107 was to update the major version referenced by "last-stable" and remove any tests that no longer made sense to test in the context of the MongoDB 3.6 release.
  • Bumping the feature compatibility version doesn't necessarily need to happen soon after we branch for a new release. For example, the changes from 67500ef as part of SERVER-29350 happened in early June 2017.

It seems like the main reason we didn't do SERVER-27355 was that at the time we weren't sure if FCV would continue to exist as a concept past 3.4. Now that we've decided to keep it as part of our general upgrade/downgrade story, it seems to make sense to add this support to our test infrastructure so that it's easier to write reusable multiVersion tests.

That was my initial apprehension to building testing infrastructure around feature compatibility version; however, Dianna closed the ticket as "Won't fix" because we ultimately deleted the tests that had explicitly set a feature compatibility version. Your request of wanting to write generic tests involving feature compatibility version changes that story. I think the simplest thing is going to be to create a module in the jstests/libs/ directory that exposes two constants.

jstests/libs/feature_compatibility_version.js

var FeatureCompatibilityVersion = {
    // To-be-updated when we bump the featureCompatibilityVersion for the next release.
    kLastStable = "3.4",
    kLatest = "3.6",
};

Then it becomes best-effort to use these constants when writing generic tests and use hardcoded values when the test isn't expected to run after we bump the feature compatibility version, as is the case with binVersion and when the test isn't expected to run after we bump "last-stable".

Comment by Spencer Brody (Inactive) [ 29/Aug/17 ]

Yeah, this came up because we recently added generic multiVersion tests to test initial syncing between the current and previous mongodb versions, and for that test to work it needs to keep the FCV in line with the previous versions. See initial_sync_last_stable_from_latest.js and initial_sync_latest_from_last_stable.js.

Those tests use the "last-stable" and "latest" labels for picking the mongod versions, but the FCV is hard coded and will need to be updated when we branch for 3.8, would be nice if they could be updated automatically.

It seems like the main reason we didn't do SERVER-27355 was that at the time we weren't sure if FCV would continue to exist as a concept past 3.4. Now that we've decided to keep it as part of our general upgrade/downgrade story, it seems to make sense to add this support to our test infrastructure so that it's easier to write reusable multiVersion tests.

Comment by Max Hirschhorn [ 29/Aug/17 ]

spencer, this idea came up during the MongoDB 3.4 release under SERVER-27355. However, after realizing that tests which run the {setFeatureCompatibilityVersion: "3.2"} command would be removed during the MongoDB 3.6 release (SERVER-27107) there wasn't any perceived value in being able to more easily specify the "last-stable" feature compatibility version. Were you planning to write a more generic upgrade/downgrade test involving feature compatibility version?

Some tests, especially in the multiversion suite, need to explicitly set the FCV to the "previous" version so that can run a mixed-version set, it would be nice to not have to locate and manually update all such tests when the major version changes.

Would adding a requires_fcv34, requires_fcv36, etc. resmoke.py tag to these tests make it easier to find these tests in the future when we go to update or remove them?

Generated at Thu Feb 08 04:25:02 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.