We should document the manual downgrade procedure for a shard server binary that attempted to downgrade to 4.4 with the secondaryDelaySecs field still present. We point users to this link: http://www.mongodb.org/dochub/core/recover-replica-set-from-invalid-config so it should be updated accordingly.
The test goes through the following procedure:
1. Start up a shard server process with MDB 4.9 / FCV (featureCompatibilityVersion) 4.4 (note: new shard servers will default to the last-LTS FCV until they are added to a sharded cluster via addShard)
2. Call rs.init() with secondaryDelaySecs in the replica set config.
3. Downgrade shard binary to MDB 4.4 / FCV 4.4 by stopping the shard server and restarting with "binary: 4.4". During this step, the shard server will refuse to start up because it sees an unrecognized field (secondaryDelaySecs). In order for the user to get out of this situation, they'll have to do the following:
- Restart shard with MDB 4.9 ("binary: 4.9") / FCV 4.4.
- Reconfig to change 'secondaryDelaySecs' to 'slaveDelay'.
- Downgrade shard binary to MDB 4.4 / FCV 4.4 by restarting shard with "binary: 4.4". The shard server should successfully start up.
Note: This is specific to downgrading to 4.4 from 5.0.