[SERVER-46089] Test downgrading a replica set with arbiters from 4.4 Created: 11/Feb/20  Updated: 29/Oct/23  Resolved: 10/Mar/20

Status: Closed
Project: Core Server
Component/s: Replication
Affects Version/s: 4.3.3
Fix Version/s: 4.4.0-rc0, 4.7.0

Type: Bug Priority: Major - P3
Reporter: William Schultz (Inactive) Assignee: A. Jesse Jiryu Davis
Resolution: Fixed Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Backports
Depends
depends on SERVER-46288 Reconfig in 4.2 style with the curren... Closed
Related
related to SERVER-45092 Remove “term” field of config documen... Closed
Backwards Compatibility: Fully Compatible
Operating System: ALL
Backport Requested:
v4.4
Steps To Reproduce:

/**
 * Test the downgrade of a replica set from latest version to last-stable version succeeds and
 * removes the "term" field of config document.
 */
 
(function() {
'use strict';
 
load('jstests/multiVersion/libs/multi_rs.js');
load('jstests/libs/test_background_ops.js');
 
let newVersion = "latest";
let oldVersion = "last-stable";
 
let nodes = {
    n1: {binVersion: newVersion},
    n2: {binVersion: newVersion},
    n3: {binVersion: newVersion}
};
 
let rst = new ReplSetTest({nodes: nodes});
let nodenames = rst.nodeList();
rst.startSet();
// rst.initiate();
 
rst.initiate({
    "_id": "remove_config_term_on_replset_downgrade",
    "members": [
        {"_id": 0, "host": nodenames[0]},
        {"_id": 1, "host": nodenames[1]},
        {"_id": 2, "host": nodenames[2], arbiterOnly: true}
    ]
});
 
let primary = rst.getPrimary();
 
// The default FCV is latestFCV for non-shard replica sets.
let primaryAdminDB = rst.getPrimary().getDB("admin");
checkFCV(primaryAdminDB, latestFCV);
 
// Reconfig in FCV 4.4.
let originalConfig = rst.getReplSetConfigFromNode();
originalConfig.version++;
reconfig(rst, originalConfig);
rst.awaitNodesAgreeOnConfigVersion();
 
// Check that the term field exists in the config document on all nodes.
rst.nodes.forEach(function(node) {
    jsTestLog("Checking the config term on node " + tojson(node.host) + " before downgrade.");
    let config = node.getDB("local").getCollection("system.replset").findOne();
    assert(config.hasOwnProperty("term"));
});
 
// Remember the config from the primary before the downgrade.
let configInNewVersion = rst.getReplSetConfigFromNode();
 
jsTest.log("Downgrading FCV to 4.2");
assert.commandWorked(primary.adminCommand({setFeatureCompatibilityVersion: lastStableFCV}));
rst.awaitReplication();
 
jsTest.log("Downgrading replica set..");
rst.upgradeSet({binVersion: oldVersion});
jsTest.log("Downgrade complete.");
 
// Check that the term field doesn't exist in the config document on all nodes.
rst.nodes.forEach(function(node) {
    reconnect(node);
    jsTestLog("Checking the config term on node " + tojson(node.host) + " after binary downgrade.");
    let config = node.getDB("local").getCollection("system.replset").findOne();
    assert(!config.hasOwnProperty("term"), tojson(config));
    config.term = configInNewVersion.term;
    assert.docEq(configInNewVersion, config);
});
rst.stopSet();
})();

Sprint: Repl 2020-03-09, Repl 2020-03-23
Participants:

 Description   

Current plan: on FCV downgrade, the primary executes a reconfig to remove the term field from the on-disk config document. The primary then blocks waiting for all nodes to replicate the new config via heartbeat. Arbiters have no special behavior in this scenario, just ensure we test this scenario in a replica set with one or more arbiters.

Original title: "Arbiters don't remove configTerm on downgrade"

Original description: Arbiters don't track FCV so they won't remove the configTerm from their on disk ReplSetConfig object on downgrade. This causes a 4.2 node to fail parsing of the config on restart after binary downgrade.



 Comments   
Comment by A. Jesse Jiryu Davis [ 10/Mar/20 ]

Removing Docs from "Teams Impacted", this ticket is now "Downstream Team Attention: None". We decided not to make arbiters special in any way.

Comment by Githook User [ 10/Mar/20 ]

Author:

{'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis', 'email': 'jesse@mongodb.com'}

Message: SERVER-46089 Test downgrade with arbiter

(cherry picked from commit ef959bbc8e0b3601323a162eb99d69716b07e548)
Branch: v4.4
https://github.com/mongodb/mongo/commit/771398011bc520c425563fa41c2096b3b361700d

Comment by Githook User [ 10/Mar/20 ]

Author:

{'name': 'A. Jesse Jiryu Davis', 'username': 'ajdavis', 'email': 'jesse@mongodb.com'}

Message: SERVER-46089 Test downgrade with arbiter
Branch: master
https://github.com/mongodb/mongo/commit/ef959bbc8e0b3601323a162eb99d69716b07e548

Comment by Siyuan Zhou [ 13/Feb/20 ]

Reopening the ticket. Even thought we don't expect behavioral change, we need to test the downgrade procedure with arbiters, so that SERVER-45091 doesn't have to address downgrade.

Comment by Siyuan Zhou [ 12/Feb/20 ]

Thanks daniel.gottlieb, that's great! I'll close this ticket as "Won't Fix".

Comment by William Schultz (Inactive) [ 12/Feb/20 ]

Ok, great, thanks for pointing that out. That might mean there's nothing to fix here.

Comment by Daniel Gottlieb (Inactive) [ 12/Feb/20 ]

Documentation states (at least as early as 4.2 -> 4.0) that downgrading arbiters requires deleting its dbpath. This was changed because the lack of a local FCV value creates headaches.

Generated at Thu Feb 08 05:10:28 UTC 2024 using Jira 9.7.1#970001-sha1:2222b88b221c4928ef0de3161136cc90c8356a66.